---
# System prepended metadata

title: my part

---

# my part


https://www.zaproxy.org/docs/docker/about/
https://semgrep.dev/docs/getting-started/


https://github.com/TheMuntu/juice-shop
https://about.gitlab.com/install/
https://docs.gitlab.com/ee/install/docker.html



## plan

1) Install gitlab
2) Semgrep
3) Owasp ZAP
4) Implementation prev solutions in my pipeline
5) Implement WAF and test if the rules are working by performing DAST.


---

## Gitlab

```dockerfile=
version: '3.6'
services:
  web:
    image: 'gitlab/gitlab-ee:latest'
    restart: always
    hostname: 'gitlab.example.com'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://localhost'
    ports:
      - '8012:80'
      - '22:22'
    volumes:
      - './config:/etc/gitlab'
      - './logs:/var/log/gitlab'
      - './data:/var/opt/gitlab'
    shm_size: '256m'
```

---

## semgrep

```bash=1
docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep scan --config=auto --json -o result.json
```

![](https://i.imgur.com/LRzbByM.png)

![](https://i.imgur.com/HBug6ex.png)

![](https://i.imgur.com/okwyDu1.png)



![](https://i.imgur.com/yDINBmW.png)


```bash=1
docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep scan --config=auto --text -o result.txt
```

![](https://i.imgur.com/uiE7xFG.png)


---


## zaproxy

Данное решение представляет возможности proxy сервера, сканера (активного и пассивного и атакующего) веб приложения а так же парсера (spider) для анализа страниц уже существующих, fuzzing решение.


Для запуска сканера необходимо описать ему окружение для тестирования и инструкцию для действий в формате yaml
https://www.zaproxy.org/docs/desktop/addons/automation-framework

example yaml file

```yaml=
--- # OWASP ZAP automation configuration file, for more details see https://www.zaproxy.com/docs/(TBA)
env:                                      # The environment, mandatory
  contexts:                               # List of 1 or more contexts, mandatory
    - name: context 1                     # Name to be used to refer to this context in other jobs, mandatory
      url: http://demo.testfire.net/             # The top level url, mandatory, everything under this will be included
      includePaths:  # TBA: An optional list of regexes to include
      excludePaths:  # TBA: An optional list of regexes to exclude
      authentication:                     # TBA: In time to cover all auth configs
  parameters:
    failOnError: true                  # If set exit on an error
    failOnWarning: true               # If set exit on a warning
    progressToStdout: true             # If set will write job progress to stdout

jobs:
  - type: addOns                       # Add-on management
    parameters:
      updateAddOns: true               # Update any add-ons that have new versions
    install:                           # A list of non standard add-ons to install from the ZAP Marketplace
    uninstall:                         # A list of standard add-ons to uninstall
  - type: passiveScan-config           # Passive scan configuration
    parameters:
      maxAlertsPerRule: 10             # Int: Maximum number of alerts to raise per rule
      scanOnlyInScope: true            # Bool: Only scan URLs in scope (recommended)
      maxBodySizeInBytesToScan:        # Int: Maximum body size to scan, default: 0 - will scan all messages
    rules:                             # A list of one or more passive scan rules and associated settings which override the defaults
  
  - type: spider                       # The traditional spider - fast but doesnt handle modern apps so well
    parameters:
      context:                         # String: Name of the context to spider, default: first context
      url:                             # String: Url to start spidering from, default: first context URL
      failIfFoundUrlsLessThan:         # Int: Fail if spider finds less than the specified number of URLs, default: 0
      warnIfFoundUrlsLessThan:         # Int: Warn if spider finds less than the specified number of URLs, default: 0
      maxDuration:                     # Int: The max time in minutes the spider will be allowed to run for, default: 0 unlimited
      maxDepth:                        # Int: The maximum tree depth to explore, default 5
      maxChildren:                     # Int: The maximum number of children to add to each node in the tree
      acceptCookies:                   # Bool: Whether the spider will accept cookies, default: true
      handleODataParametersVisited:    # Bool: Whether the spider will handle OData responses, default: false
      handleParameters:                # Enum [ignore_completely, ignore_value, use_all]: How query string parameters are used when checking if a URI has already been visited, default: use_all
      maxParseSizeBytes:               # Int: The max size of a response that will be parsed, default: 2621440 - 2.5 Mb
      parseComments:                   # Bool: Whether the spider will parse HTML comments in order to find URLs, default: true
      parseGit:                        # Bool: Whether the spider will parse Git metadata in order to find URLs, default: false
      parseRobotsTxt:                  # Bool: Whether the spider will parse 'robots.txt' files in order to find URLs, default: true
      parseSitemapXml:                 # Bool: Whether the spider will parse 'sitemap.xml' files in order to find URLs, default: true
      parseSVNEntries:                 # Bool: Whether the spider will parse SVN metadata in order to find URLs, default: false
      postForm:                        # Bool: Whether the spider will submit POST forms, default: true
      processForm:                     # Bool: Whether the spider will process forms, default: true
      requestWaitTime:                 # Int: The time between the requests sent to a server in milliseconds, default: 200
      sendRefererHeader:               # Bool: Whether the spider will send the referer header, default: true
      threadCount:                     # Int: The number of spider threads, default: 2
      userAgent:                       # String: The user agent to use in requests, default: '' - use the default ZAP one
  - type: passiveScan-wait             # Passive scan wait for the passive scanner to finish
    parameters:
      maxDuration: 5                   # Int: The max time to wait for the passive scanner, default: 0 unlimited
  - type: activeScan                   # The active scanner - this actively attacks the target so should only be used with permission
    parameters:
      context:                         # String: Name of the context to attack, default: first context
      policy:                          # String: Name of the scan policy to be used, default: Default Policy
      maxRuleDurationInMins:           # Int: The max time in minutes any individual rule will be allowed to run for, default: 0 unlimited
      maxScanDurationInMins:           # Int: The max time in minutes the active scanner will be allowed to run for, default: 0 unlimited
      addQueryParam:                   # Bool: If set will add an extra query parameter to requests that do not have one, default: false
      defaultPolicy:                   # String: The name of the default scan policy to use, default: Default Policy
      delayInMs:                       # Int: The delay in milliseconds between each request, use to reduce the strain on the target, default 0
      handleAntiCSRFTokens:            # Bool: If set then automatically handle anti CSRF tokens, default: false
      injectPluginIdInHeader:          # Bool: If set then the relevant rule Id will be injected into the X-ZAP-Scan-ID header of each request, default: false
      scanHeadersAllRequests:          # Bool: If set then the headers of requests that do not include any parameters will be scanned, default: false
      threadPerHost:                   # Int: The max number of threads per host, default: 2
    policyDefinition:                  # The policy definition - only used if the 'policy' is not set
      defaultStrength: Medium    # String: The default Attack Strength for all rules, one of Low, Medium, High, Insane (not recommended), default: Medium
      defaultThreshold: Medium  # String: The default Alert Threshold for all rules, one of Off, Low, Medium, High, default: Medium
      rules:                           # A list of one or more active scan rules and associated settings which override the defaults
      - id: 40012
```

```
The framework supports:

environment - which defines all of the applications the plan can act on
Authentication - all of the authentication mechanisms supported by ZAP
Job Tests - which can be used to validate the outcome of jobs
The full set of jobs currently supported by the framework and other add-ons are:
```

```
activeScan - runs the active scanner
alertFilter - alert filter configuration, provided with the Alert Filters add-on
delay - waits for a specified time or until a condition is met
graphql - GraphQL schema import, provided with the GraphQL add-on
import - allows you to import HAR(HTTP Archive File), ModSecurity2 Logs, ZAP Messages or a file containing URLs locally
openapi - OpenAPI definition import, provided with the OpenAPI add-on
passiveScan-config - passive scan configuration
passiveScan-wait - waits for the passive scanner to finish processing the current queue
report - report generation, provided with the Report Generation add-on
requestor - sends specific requests to targets
script - adds, removes and runs scripts
soap - SOAP WSDL import, provided with the SOAP add-on
spider - runs the traditional spider
spiderAjax - runs the ajax spider, provided with the Ajax Spider add-on
```

```bash=1
docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml
```

```bash=1
docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable bash -c "zap.sh -cmd -addonupdate; zap.sh -cmd -autorun /zap/wrk/zap.yaml"
```

```bash=1
docker container run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-weekly zap.sh -cmd -autorun /zap/wrk/zap.yaml
```

---



## DefectDojo

```bash=1
docker run -it --rm  -v $(pwd):/ssl -p 8000:8000 appsecpipeline/django-defectdojo bash -c "export LOAD_SAMPLE_DATA=True && export ALLOWED_HOSTS="defect.lichi.su" && bash /opt/django-DefectDojo/docker/docker-startup.bash"

```

```yaml=
dojo:
  image: defectdojo/dojo
  script:
    - dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f zap-report.xml -t "ZAP Scan"
    - dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f dependency-check-report.xml -t "Dependency Check"
    - dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f gitleaks-secret-detection.json -t "Gitleaks"
    - dojo -E $DOJO_ENVIRONMENT -U $DOJO_USER -K $DOJO_API_KEY -P $DOJO_URL -n "My Project" -p "My Product" import_scan -f semgrep-sast-report.json -t "Semgrep"
  dependencies:
    - dast
    - dependency_check
    - secret_detection
    - sast
  tags:
    - docker
```

https://www.oreilly.com/library/view/practical-security-automation/9781789802023/7a780273-63e9-404f-9075-7292191de8ce.xhtml


---

## Gitlab


https://docs.gitlab.com/ee/install/docker.html



Final pipeline

```yaml=
variables:
  DOCKER_REPO_NAME: "juice-shop"
  ANSIBLE_IMAGE: "ansible/container-conductor-alpine-3.5:0.9.3rc4"
  KANIKO_IMAGE: "gcr.io/kaniko-project/executor:debug"


stages:
  - sast_semgrep
  - build
  - dast_zap
  - dojo


.build-only: &build-only
  only:
    - master
    - tags

sast_job:
  stage: sast_semgrep
  image: docker:20.10.16
  services:
  - docker:20.10.16-dind
  script: 
    - docker pull returntocorp/semgrep
    - docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep scan --config=auto --json -o semgrep-sast-report.json
  artifacts:
    paths:
      - semgrep-sast-report.json

dast_job:
  stage: dast_zap
  image: docker:20.10.16
  services:
  - docker:20.10.16-dind
  script: 
    - docker pull owasp/zap2docker-stable
    - docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun /zap/wrk/zap.yaml
  artifacts:
    paths:
      - dast-report.html

build_job:
  stage: build
  <<: *build-only
  image:
    name: $KANIKO_IMAGE
    entrypoint: [""]
  script:
    - echo "Run image building process"
    - echo "{\"auths\":{\"$DOCKER_REGESTRY_URL\":{\"username\":\"$DOCKER_REGESTRY_USERNAME\",\"password\":\"$DOCKER_REGESTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
    - >
      if [ ! -z $CI_COMMIT_TAG ]; then
      /kaniko/executor
      --context $CI_PROJECT_DIR
      --dockerfile $CI_PROJECT_DIR/$DOCKERFILE
      --destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:$CI_COMMIT_TAG
      --destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:$CI_COMMIT_SHORT_SHA
      --destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:latest
      --cache=true; else
      /kaniko/executor
      --context $CI_PROJECT_DIR
      --dockerfile $CI_PROJECT_DIR/$DOCKERFILE
      --destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:$CI_COMMIT_SHORT_SHA
      --destination $DOCKER_REGESTRY_URL/$DOCKER_REGESTRY_PROJECT/$REPO_NAME:latest
      --cache=true; fi
  variables:
    DOCKERFILE: Dockerfile
    REPO_NAME: $DOCKER_REPO_NAME

dojo_sender_job:
  stage: dojo
  image: $DOCKER_REGESTRY_URL/999669/dojo_sender:latest
  script:
    - sh -c "php /app/index.php '$DOJO_HOST' '$DOJO_TOKEN' '$DOJO_LEAD' 'Semgrep JSON Report' 2 'semgrep-sast-report.json'"
    - sh -c "php /app/index.php '$DOJO_HOST' '$DOJO_TOKEN' '$DOJO_LEAD' 'ZAP Scan' 2 'dast-report.html'"

```


ZAP
```yaml=
--- # OWASP ZAP automation configuration file, for more details see https://www.zaproxy.com/docs/(TBA)
env:                                      # The environment, mandatory
  contexts:                               # List of 1 or more contexts, mandatory
    - name: context 1                     # Name to be used to refer to this context in other jobs, mandatory
      urls:
        - https://farm.stageogip.ru
      sessionManagement:
        method: "cookie"
        parameters: { }
      users:
        - name: "cxadmin"
          credentials:
            password: "43VcKY8Hg567i2wJ@"
            username: "admin"
  parameters:
    failOnError: true                  # If set exit on an error
    failOnWarning: true               # If set exit on a warning
    progressToStdout: true             # If set will write job progress to stdout

jobs:
  - type: passiveScan-config           # Passive scan configuration
    parameters:
      maxAlertsPerRule: 10             # Int: Maximum number of alerts to raise per rule
      scanOnlyInScope: true            # Bool: Only scan URLs in scope (recommended)
      maxBodySizeInBytesToScan:        # Int: Maximum body size to scan, default: 0 - will scan all messages
    rules: # A list of one or more passive scan rules and associated settings which override the defaults
      - id:                              # Int: The rule id as per https://www.zaproxy.org/docs/alerts/
        name:                            # String: The name of the rule for documentation purposes - this is not required or actually used
        threshold:                       # String: The Alert Threshold for this rule, one of Off, Low, Medium, High, default: Medium

  - type: spider                       # The traditional spider - fast but doesnt handle modern apps so well
    parameters:
      failIfFoundUrlsLessThan:         # Int: Fail if spider finds less than the specified number of URLs, default: 0
      warnIfFoundUrlsLessThan:         # Int: Warn if spider finds less than the specified number of URLs, default: 0
      maxDuration:                     # Int: The max time in minutes the spider will be allowed to run for, default: 0 unlimited
      maxDepth:                        # Int: The maximum tree depth to explore, default 5
      maxChildren:                     # Int: The maximum number of children to add to each node in the tree
      acceptCookies: true                  # Bool: Whether the spider will accept cookies, default: true
      handleODataParametersVisited:    # Bool: Whether the spider will handle OData responses, default: false
      handleParameters:                # Enum [ignore_completely, ignore_value, use_all]: How query string parameters are used when checking if a URI has already been visited, default: use_all
      maxParseSizeBytes:               # Int: The max size of a response that will be parsed, default: 2621440 - 2.5 Mb
      parseComments:                   # Bool: Whether the spider will parse HTML comments in order to find URLs, default: true
      parseGit:                        # Bool: Whether the spider will parse Git metadata in order to find URLs, default: false
      parseRobotsTxt:                  # Bool: Whether the spider will parse 'robots.txt' files in order to find URLs, default: true
      parseSitemapXml:                 # Bool: Whether the spider will parse 'sitemap.xml' files in order to find URLs, default: true
      parseSVNEntries:                 # Bool: Whether the spider will parse SVN metadata in order to find URLs, default: false
      postForm: true                       # Bool: Whether the spider will submit POST forms, default: true
      processForm: true                    # Bool: Whether the spider will process forms, default: true
      requestWaitTime:                 # Int: The time between the requests sent to a server in milliseconds, default: 200
      sendRefererHeader:               # Bool: Whether the spider will send the referer header, default: true
      threadCount:                     # Int: The number of spider threads, default: 2
      userAgent:                       # String: The user agent to use in requests, default: '' - use the default ZAP one

  - type: activeScan                   # The active scanner - this actively attacks the target so should only be used with permission
    parameters:                        # String: Name of the context to attack, default: first context
      policy:                          # String: Name of the scan policy to be used, default: Default Policy
      maxRuleDurationInMins:           # Int: The max time in minutes any individual rule will be allowed to run for, default: 0 unlimited
      maxScanDurationInMins:           # Int: The max time in minutes the active scanner will be allowed to run for, default: 0 unlimited
      addQueryParam:                   # Bool: If set will add an extra query parameter to requests that do not have one, default: false
      defaultPolicy:                   # String: The name of the default scan policy to use, default: Default Policy
      delayInMs:                       # Int: The delay in milliseconds between each request, use to reduce the strain on the target, default 0
      handleAntiCSRFTokens:            # Bool: If set then automatically handle anti CSRF tokens, default: false
      injectPluginIdInHeader:          # Bool: If set then the relevant rule Id will be injected into the X-ZAP-Scan-ID header of each request, default: false
      scanHeadersAllRequests:          # Bool: If set then the headers of requests that do not include any parameters will be scanned, default: false
      threadPerHost:
  - type: report
    parameters:
      template: traditional-json-plus
      displayReport: true
      reportDir: /zap/wrk
      reportFile: dast-report.json
    risks: # List: The risks to include in this report, default all
      - high
      - medium
      - low
      - info
    confidences: # List: The confidences to include in this report, default all
      - high
      - medium
      - low
      - falsepositive
    sections:                          # List: The template sections to include in this report - see the relevant template, default all

```