Crowd sourced TESTNET evalution for STEEM

High Level structure of the document:
- what - why - how - who
# what

gandalf 1:54 PM
Try to stay up to date with the code on the github. Testing is not trivial. And it's needed not only when hardforks come. ( https://steemit.com/@gtg)

TODO: Clear problem defintion

# why

Problem specificiation

Unlike other public blockchains, STEEM blockchain is the first blockchain to interact with end users who are not necessarily tech savvy or traders. This blockchain creates value by incentivicing content creators. To keep the platform running frequent changes to the code base is required to meet the ever changing demands in terms of bug fixes, prevent ways of abusing the platform & add new features. While the core blockchain consisting of the consensus (DPoS), storing/capturing the state etc are mature, the plugins which adds additional functionalities for the end users to interact with the blockchains in evolving at an unprecedented pace. The plugins which are changing includes, rc_plugin (Resource credits + Turing Compelete), reputation, SMT (new tokens, smart contracts) support etc. The amount of changes at this enormity are perhaps unprecedented in the history of blockchains and thus, end to end testing is needed to ensure maximum code coverage and crush the obvious bugs and gremlins.

# how

This section will define the solution

- Define KPI for each minor and major relase (build time, TPS, replay time, space/time)
- automatic static analysis
- code review (already done internally)
- continous integration
- Continous Deployment when soft/hard forks approach
- Alert witnesses about the changes required (in config files, replay needed, gcc/clang, boost etc)
- mobilize community to interact with TESTNET
- Generate Transactions using Tinman
- Testing of Condensor, a/c creation faucet, condensor, SSO, steem-js, steem-python, steem-ruby
- Co-ordinate with SIAB / beem / busy / steempeak / steemjs-tools / conductor / st
- Create status page, telegram groups etc for the TESTNET + MAINNET
- CHANGELOG, Impact etc must be published adhering to a schedule
- The community will be giving feedbacks to Developers to support the existing development process first and then if needed to improve it.
- The community should not get in the way of development, but MUST act like a "gate keeper"
- Existing tools used by the developers should be used to maximize the impact as opposed to adding new tools
- Security issues or high severity bugs should be communicated only to designated developers via secure means

*TODO:* Add more points from witness testaments (refer @[timcliff](https://steemit.com/@timcliff)'s [post](https://steemit.com/witness-category/@timcliff/the-reports-from-the-witnesses-2018-09-23)
https://steemit.com/witness-update/@reggaemuffin/witness-statement-for-reggaemuffin-proposing-hardfork-adoption-requirements
*TODO:* Add more points from community feedback

@yuriks2000
1. suggested using conductor to connect to the testnet for bringing end users to test the new fork
2. 05.10.2018 Set up https://steemtest.com/ especialy for testing new hardforks
3. https://steemtest.com/ will look at the steemd of @bobinson server with the new versions or hardforks of Steem

@quochuy - has a script (?) to take the state snapshot

@gtg
It's useful for big infrastructure, for example, if I have 5 edge nodes on prod running `v0.20.4` and I wan't to switch them ASAP to replayed version of `v0.20.5` then I shutdown one of them, lets say `edge5` and start `--replay` (because I don't need `--resync` since I had already valid `block_log`). After catching up with HEAD block (replay to last block in log, then sync to current HEAD) I shut down `edge5` and make a copy of state, that is: `block_log`, `block_log.index` and `shared_memory.bin`. I start `edge5` back up, shutdown `edge4`, copy state file there and start `edge4` back up with the new version and new state.

# Transaction Generation on the STEEM TESTNET

**"A blockchain is a Distributed OpenLedger of records, contracts & Transactions between multiple partieis."**

An ideal TESTNET must sync Records, Contracts & Transactions in real time thus replicating the state of the MAINNET.

Further this can be summarized as state transfer from one distributed state machine to another resulting in invocation of various autmatic transctions and virtual operations. In a real world scenario, this will not be possible and we will have device elaborate mechanisms to mimic real world interactions.

## discussion

List down the transaction types that can be invoked with a tool similar to Tinman. For these cases, including a periodic sync of state from the MAINNET to the TESTNET can be done with Tinman. Further mimicing the user interaction, reaching the TPS that we can expect on the MAINNET etc are going to be the challenges. If we are syncing from the MAINNET it will not be possible sync the current Resource Credits (or earlier bandwidth data). We will be able to get the posts. For the interactions on the posts we will need to device a mechanism and invite the community.

## What is achieved so far

1. mirror from github to gitlab
2. CI on Kubernetes is configured
3. Deploy to a testnet with 3 nodes is done (with stale production)
4. Replayed blockchain related files (v0.20.5) are copied (via EBS snapshots)
5. Tinman is auto deployed and triggered from gitlab but not tested against TESTNET
6. Condenser set up on https://steemtest.com/
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ That's a really bad idea to run testing enviornment without clearly warning users about it. Lets change Steemit logo to something that clearly indicates test environment.

# who

Stake holders will be here. This will be generally "we"

References

1. https://steemit.com/witness-update/@reggaemuffin/witness-statement-for-reggaemuffin-proposing-hardfork-adoption-requirements

# work arounds

Since there are changes to bandwidth, here is a trick to test the replay