# External policy provider

###### tags: `External policy provider`

> HackMD specification for **external policy provider**
> - :arrow_right: [PR with old implementation](https://github.com/ArweaveTeam/arweave/pull/207/files)
> - :arrow_right: [PR with new implementation](https://github.com/ArweaveTeam/arweave/pull/243)

**Topics:**
[ToC]

# :memo:Goals
Need to implement logic for external policy provider for server side.

# Implementation

- [x] Update logic for configuration
- [X] Support of basic authorisation
- [x] Create parser for files with TXs
- [x] Create API for download files with TXs
- [x] Implement gen_server
- [X] Implement logic for cleanup
- [X] Test coverage
- [x] UML diagrams

:rocket:

## Update logic for configuration

Users should have posability download and store links with TXs for external policy provider by configuration. The name of new configuration parameter is `content_policy_url`. The URL can have two formats:
- `https://some.url/file.txt`
- `https://username:[e-mail]/file.txt`
:::info
:pushpin: **Must have:** need support to put multi URLs into `content_policy_url` config:

$ ./arweave-server content_policy_url https://some.url/file.txt https://username:[e-mail]/file.txt

:::

## Support of basic authorisation
When user put URL like `https://username:[e-mail]/file.txt` this will be mean that the REST call to URL `some.url/file.txt` should have headers in request `Authorization: Basic base64(username:password)`.

## Create parser for files with TXs

The file should have sets of transactions with standard base64 style encoding. Delimiter is `\n`, however the delimiter also can be added any other whitespace chars by user or service mistakes. Need create parser which will parse `.txt` file with whitespaces.

| Whitespace | Description |
| ----------------- |:----------------------- |
|`\n` | New line |
|`\t` | Tabulations |
| etc. | ... |

## Create API for download files with TXs
Need create API for download files by `gun` library. In current API should call parser for each file what will be downloaded and parsed.
:::info
:pushpin: **Must have:** in the background should call `gen_server` for cleanup transactions by external policy provider
:::

## Implement gen_server
Need to implement `gen_server` with logic:

- Call in background the cleanup functionality with `N` interval
- Call in background the cleanup functionality after each confifuraticonfigurationon updates.

## Implement logic for cleanup
- [X] Cleanup transactions by external policy provider data
- [X] Cleanup transactions data in `ar_data_sync`

## Test coverage
Need implement logic by tests for:
- Create mock file server
- Emulation behavior of storing configuration with new parameter
- Emulation behavior for donwload and parse files from services with storing transactions in RocksDB and with call of `gen_server` logic. Maybe need to use `permapaste` or `http://ptsv2.com` for test downloading files
- Emulation behavior for recall of `gen_server` logic for each config updating

## UML diagrams

Shell --> API: 
Note left of Shell: Ok
API->Shell: Download and store content policy by URLs
API-->gen_server:
gen_server->API:
gen_server-->cleanup:
cleanup->gen_server: 
Note right of cleanup: Ok