Idea for the client/server interaction:
=======================================

# Goals:

- Make the client thinner so there's less redundancy between client&server -> less bugs, and more user control from backend side -> more flexibility when stuff gets rough.

- More systematic -> so client/server API becomes easier to expand and understand -> overall system complexity reduced.

- More flexible -> faster to change and develop.

- Possible to gradually transform into a push-api through web-sockets (and eventually remove pulls!) --> faster and more flexible (in case of bug or customer support interaction, ability to modify a response to call ... even if the call wasn't done!)

- Easier to mock our backend -> so backend devs would not block frontend devs -> also easier to automate tests.

- Explicit representation of frontend state:
- -> customer support team will be able to see the frontend state in real time.
- -> while debugging (or for business intelligence) it will be possible to access to the history of each users frontend state.
- -> in emergencies , it will be possible to test/check how the code would behave given an actual real users state at a given point in time!
(of course this would require privileged db production keys).

# Introduction to the propossal:

The propossal main points are:

- On the frontend side keep a React approach as it is now, in fact, keep as much as possible as it is now.

- Split the state so that it is transitional (it dissapear if you F5), away from the persistent state, and represent it explicitely with react components.

- Persistent state might affect transient state, but not vice-versa.

- The persistent state, with the exception of the cookies, is represented only by the PersistentState component and is clearly separated from the rest of the state of the UI.

- The transient state might be mostly represented on the TransientState component, but other U.I component might have their own state so the way that the transient state is represented is something more lenient and flexible; so we can keep most of our current implementation of the.

- The PersistentState component would be the only source of persistent state for the app, this means:

- No firebase.

- Calls to the backend would return only success (`{}`) or error (`{error_code:'ERR_CODE_NOT_FUND', debug_info: null, translated_error_msg:{es:'No te queda dinero', fr:'Pas €!'}`)

- The exception would be a call to a special backend-function `getPersistentState` that will return the frontend persistent-state.

- This function will be possible to call through polling or when the frontend thinks there might be a change in persistent-state. Eventually we will use websockets instead of polling and the backend would push the PersistentState to the client every time there's a change to PersistentState.

- From the frontend point of view, there would only be 1 "backend", no distinction between onramp/onsis.

- TODO: what about other services like bitcoin clients? Maybe they could be an exception to these ideas? What are the tradeoffs?

- The PersistentState component representation (as Json) will be **identical** to the value returned from the `getPersistentState` function/push-message.

- This PersistentState would work as the interface between backend developers and frontend developers, and is expected to be extended by both.

- Therefore, the format and the way it should be extended will follow the some specific rules, specified in a later section of this document (See PersistentState format).

- It will be easy to be extended, both for frontend and backend devs, and it will be more easily understandable by customer support.

- For development, debugging or crisis management, there would be a mock-backend.

- For every request except `getPersistentState`, the mock will return `{}`.

- For `getPersistentState` the mock will return the ("jsonify") content from local file `mockPersistentState.yaml`.

- The idea is that a frontend developer will be able to try any interaction not yet implemented on the backend,
by just writing down in his editor, and pretending to be the server, editing any change he plans to implement.

- Once push notifications through web-sockets are working, then every time a change on disk on for a specific file is detected, the mock will send the content of that file to the client.

- For production crisis or debugging, we could query the DB to find the persistent state of a given user, paste it to `mockPersistentState.yaml`,
find the frontend code version in the logs, and then we will be able to see what the user sees.

- In the future, with a GUI and some interaction with the actual backend, it could be used to show customer support what the user sees.

- In the future, using regex, it could also emulate some 4XX error, defined on a file `mockErrors.yaml` , such as:

      # if the call endpoint and post_body content fits the endpoint regex and post_body regex from any of the entries here,
      # return the given error
      error_foo:                              
        endpoint: /rpc/push_receipt/*/ 
        post_body:
          receipt_id: *
          user_id   : test_user
        error:
          error_code: ERR_CODE_BAD_RECEIPT
          error_msg:
            es: Recivo incorrecto
            en: Bad Receipt
            fr: Receive ce ne pais correct.

      not_allowed_user:                              
        endpoint: /rpc/push_receipt/*/ 
        post_body:
          user_id   : not_log_user
        error:
          error_code: ERR_CODE_BAD_RECEIPT
          error_msg:
            es: Unknown user
            en: Behold hacker!
            fr: ok

# Flow diagram:

 #=====================================================================================================#
 ||                                                                                                   ||
 ||                                          B A C K E N D                                            ||
 ||                                        B L A C K   B O X                                     ^    ||
 ||          |                                                                                   |    ||
 ||          |                                                                                   |    ||
 ||          |                                                                                   |    ||
 #===========|===================================================================================|=====#
             |                                                                                   |
             |                                                                          is ok?   |          is 4XX error?
             |                                                                                   |       
             |                                                                              POST requests-----------------*
       Either response                                                                           |                        |
        from polling                                                                             |                        |
             or                                        *-----------------------------------------*           Already translated, human readable
       websocket push                                  |                     |                   |              error to be "pop-upped"
             |                                         |                     |             users clicks a                 |
             |                                         |                     |                button                      |
             |                                         |                     |                   |                        |
             V                                         v                     |                   |                        |
   +-------------------+                      +-------------------+          |           +-------------------+            |
   |                   |                      |                   |          *---------->|                   |            |
   |                   |                      |                   |                      |       U.I.        |            |
   | Persistent State  |---Reacts cascades--->| Transient State   |---Reacts cascades--->|                   |<-----------/
   |                   |    a change down     |                   |    a change down     | (Some components  |
   |                   |                      |                   |                      |    might be       |
   |                   |                      |                   |                      |    Stateful )     |
   +-------------------+                      +-------------------+                      +-------------------+
                                                       ^ 
                                                       |
                                                       |
                                              change on browser's URL
                                                or F5

# PersistentState format and extensions.

To keep `PersistentState` systematic so backend-devs, frontend-devs and customer-support get familiar with it and understand it even if it is continuosly changing, it will be auto-generated based on the events issued by the backend (More on this on the backend change proposal): Backend-devs (or even frontend-devs) will add new events and then they'll appear on `PersistentState`.

The top level of `PersistentState` is a key-value dictionary, where the key is the name of the type of event, and the value is a list of the events of that type, sorted from most recent to oldest.

- Examples of events types names would be `credit_card_rejected`, `merchant_funds_received`, `egress_approved` ... etc.

- Events are jsons objects with the following fields:

- `event_payload`: A json object with fields for the specific type of events.

- `log_entry`: If this events must be shown on the user-log-entry, then this will contain the information required to create the entry, right now that would be an already translated text (i.e `{es: '¡Hola!', fr: 'Bonjour!', en: 'Greetings!'}`). If the event should not be shown on the log-entry, then this would be `null`.

- `issued_at`: When the event was created, not necesarilly when the user received it.

- `event_id`: A unique uuid for this event, mainly for debugging propporses.

- `target`: The list of communication channels the event comes from.

- `process`: Information about the process that issued the events. This is required to be able to associate events to eachother. For example, if you are attempting to pay 2 different invoices at the same time, and you get an event "invoice_cancel", you need to know from which process this came from in order to know which of the 2 invoices is actually cancelled. The fields are:

- `id`: A unique value (could be a json object) that identifies the process. (this is kind of a generalization of the `op_id`).
- `name`: What type of process it was (i.e `operation_l2coin_to_merchant_balance`).

An example of what it would look like:

{ persistent_state:
     { new_utxo :
         [ { process:
                { name: 'piece_transaction'
                , id: «txid»
                }
           , target:
                [ {pubkey : «pubkey»}
                ]
           , event_issued : '2020-01-30 14:00 UTC'
           , login :
               { en : 'Youve got 3245 starpoints'
               , es : 'Conseguiste 3245 starpoints'
               }
           , event_payload:
              { amount : 3245
              , token: 'starpoints'
              }
           }
         , { process:
                { name: 'piece_transaction'
                , id: «txid»
                }
           , target:
                [ {pubkey : «pubkey»}
                ]
           , event_issued : '2020-01-30 14:00 UTC'
           , login :
               { en : 'Youve got 3245 starpoints'
               , es : 'Conseguiste 3245 starpoints'
               }
           , event_payload:
              { amount : 3245
              , token: 'starpoints'
              }
           }
         ]
     , spent_utxo:
         [ { process:
                { name: 'piece_transaction'
                , id: «txid»
                }
           , target:
                [ {pubkey : «pubkey»}
                ]
           , event_issued : '2020-01-30 14:00 UTC'
           , login :
               { en : 'Youve got 3245 starpoints'
               , es : 'Conseguiste 3245 starpoints'
               }
           , event_payload:
              { amount : 3245
              , token: 'starpoints'
              }
           }
         ]
     , payment_start :
         [ { process:
               { name: 'operation_credit_card_to_l2coin'
               , id  : '000000-0000-0000-000000'
               }
            , target:
                [ {access_id: '000000-0000-0000-000000'}
                , {email: '[e-mail]'}
                ]
           , event_issued: '2020-01-30 14:00 UTC'
           , event_id: '000000-0000-0000-000000'
           , login: null
           , event_payload:
              { amount_to_paid_in_cent: 1234
              , fiat_currency: 'EUR'
              }
           }
         ]
     }
}

For the same combination `process.id + process.name + type_of_events`, it will only keep the last `K` instances. This means most events will be kept forever, because usually for the same process (i.e `op_id`) each type of event is triggered only once; but some other events, like `price_update` or `fee_changed` will happen more frequently, and only the most recent ones will be returned.

# Communication Channels:

An event could be sent to more than an user (i.e change on fees), and could reference a user in different ways, (i.e user with `access_id=xxx`, user with `cookie=yyyy`, user with `pubkey=zzzz`).
These different ways will be called _communication channels_. When a user want to request `getPersistentState` , it will specify a list of communication channels, for example:

curl -X Post  localhost/rpc/getPersistenState '{access_id = 'this', email = '[e-mail]', fee_channel = true }'

Communication channel based on cookies will be set by default unless specified.

# Gradual implementation:

Instead of making a big change, the idea is to implement just the minimal foundation, and move gradually from there. Once the structure is there, i.e, calls to `getPersistentState`, the `PersistenState` component, we could keep all the previous logic, calls and code; but every new call will return its value through the `PersistenState` rather than directly. In addition to that, step by step, previous calls will be changed to ignore the result, and use only the one from `PersistentState`...with a bit of luck, eventually `PersistenState` would represent the actual persistent state.

# Tasks:

TODO: dependencies between the task.

TODO: Further definition of these:

- On errors, if the field `translated_error_msg` is returned, then instead of calling back for translated the error code, use `translated_error_msg` directly.

- Create an initial trivial PersistentState and reorganize the react component tree to contain it.

- Plug the getPersistentState to the current polling mechanism.

- Plug the getPersistentState to the rpc call, such that if any call returns a field name `persistent_state`, that will be used as the persistent state. (i.e, the client calls the backend, and in addition to the fields it will normally return, there might be eventually one named `persistent_state`). The idea of this is to be able to migrate to persistent-state usage in such way that it does not imply more calls to the backend and hence slowing down UX.

- Implement the mock-backend.

- Start using `getPersistentState` to substitute calls to `getReceipts`

- Start using `getPersistentState` to remove dependencies to the `status` field on those problematic cases.

- Once the cookies communication channels is working, and the backend authentication endpoints have been enhanced to support it; use only backend authentication methods and drop firebase.

# Maintenement & quality improvement:

Once the minimal foundation is there, it will be key to maintain a background, slow but continious, refactoring process. Maybe the best idea for this is for the devs that will be involved to decide and agree on some set of clear rules that would ensure that the refactoring process will not stop and will advance gradually. For example, it could be decided that every PR or every release would need to do some small refactor work (i.e, stop using some fields returned from a call, start using something from `PersistentState`). In addition to that, it would be great if somebody from the frontend guys could take responsability of checking that those rules are kept on the PR (this could be for example a rotating role, or something to do once after release ... etc) and that this role will create backend tasks about removing things once they are no longer needed on the frontend side.