# Refactoring Resolvers and Data Models in Cenzontle

## Goals

* Fulfill the keep it simple and safe, and the don't repeat yourself principles.
* Perform authorization checks on both ends of an association.
* Check for record limits when creating and updating associations.

## Design sketches

We move everything that can be moved from the data model layer into the resolver layer.

### Example Resolver Module

// person resolver 
const associationArgsDef =  {
  'addDogs': 'Dog', 
  'removeDogs': 'Dog', 
  'addParrots': 'Parrot',
  'removeParrots': 'Parrot',
  'addEmployer': 'Employer', 
  'removeEmployer': 'Employer'
}

// root mutation (resolver)
// e.g. 
addPerson( input, context ) {
  let inputSanitized = sanitizeAssociationArguments( input, /* assocArgNames */ )
  // Security check
  checkAuthorization(...) // standard create permission on "Person" model as before
  checkAuthorizationOnAssocArgs( inputSanitized, context, associationArgsDef )
  // check if association args don't exceed record limit
  checkAndAdjustRecordLimitForCreateUpdate( inputSanitized, context, associationArgsDef )
  // check if association args actually point to existing IDs
  // MAKE SURE THAT IN CASE OF A REMOTE DATA MODEL (DDM, cenzontle-web-server, generic web-service)
  // that the association related args in input are ignored or removed, i.e. not processed.
  // MAKE SURE THE 
  validateAssociationArgsExistence( inputSanitized, context, 
  // HANDLE PERSON ATTRIBUTES
  // models.Person.addOne uses named function arguments, to
  // only receive Person scalar attributes
  let theNewBaby = models.Person.addOne( sanitizedInput )
  // In Case of the DDM the above line passes the input the a responsible adapter!
  // HANDLE ASSOCIATIONS
  theNewBaby.handleAssociations()
}
 
updatePerson( input, context ) {
  let inputSanitized = sanitizeAssociationArguments( input, /* assocArgNames */ )
  // Security check
  checkAuthorization(...) // standard update permission on "Person" model as before
  checkAuthorizationOnAssocArgs( inputSanitized, context, associationArgsDef )
  // check if association args don't exceed record limit
  checkAndAdjustRecordLimitForCreateUpdate( inputSanitized, context, associationArgsDef )  
  // check if association args actually point to existing IDs
  validateAssociationArgsExistence( inputSanitized, context, associationArgsDef )
  // HANDLE PERSON ATTRIBUTES
  // models.Person.updateOne uses named function arguments, to
  // only receive Person scalar attributes
  let theOldFart = models.Person.updateOne( inputSanitized )
  // HANDLE ASSOCIATIONS
  theOldFart.handleAssociations()
}

// Person DATA MODEL LAYER
// ***********************
// Uses NAMED FUNCTION ARGUMENTS to
// ONLY receive Person attributes
addOne( {name, age, email, haircolor} ) {
  return // result of write into storage
}

### Helper functions

// graphql-server/utils/helper.js
// ******************************

removeAssociationArgs(input, associationArgsDef) {
   let clonedInp = /* clone input */
   Object.keys(associationArgsDef).forEach(key => {
     delete clonedInp[key]
   })
   return clonedInp
 


checkAuthorizationOnAssocArgs( input, context, associationArgsDef, permissions = ['read', 'update'] ) {
  
  // THIS FUNCTION SHOULD RETURN TRUE if and only if no authorization check failed.
  // Throw the respective error(s) if authorization fails.
  Object.keys(associationArgsDef).reduce( function(acc, curr) {
    let hasInputForAssoc = isNonEmptyArray(input[curr]) || isNotUndefinedAndNotNull(input[curr])
    if (hasInputForAssoc) {
      let targetModelName = associationArgsDef[curr]
      // Look into the definition of the associated data model and ask for its storage type.
      // TWO CASES: 
      // 1) target model storage type: NON distributed (any other)
      return permissions.reduce( (acc, curr) =>
        acc && helper.checkAuthorization(context, targetModelName, curr ),
        true
      )
      // 2) target model storage type: distributed model (DDM)
      // Get mathematical set of responsible adapters for Ids in input
      // check 'permissions' on these adapters
      // Difference to above is getting Adapters for provided association IRIs (IDs)
      // and check the argument permissions on each of those
    } else {
     return acc
    

  }, true)



checkAndAdjustRecordLimitForCreateUpdate( input, context, associationArgsDef ) {
  let totalCount = helper.countRecordsInAssociationArgs( 
    input, context.record_limit, 
    Object.keys(associationArgsDef)
  )
  // add one to total count, to reflect the "root" record being created or updated:
  totalCount++
  if (totalCount > context.recordLimit) { throw new Error(...) }
  // adjust record limit
  context.recordLimit -= totalCount
  return totalCount
}
async validateAssociationArgsExistence( input, context, associationArgsDef ) {
  Object.keys(associationArgsDef).reduce( 
       function(prev, curr) {
    let acc = await prev;
    let currAssocIds = input[curr]
    // check if set - mind that it can be either an Int or an Array!
    if (/* currAssocIds is null or undef or empty array */) { 
      return acc && true 
    } // else:
    // if not array, make it one
    if (! isNonEmptyArray( currAssocIds ) ) { currAssocIds = [ currAssocIds ] }
    // do the check
    let currModel = associationArgsDef[curr]
    let idsNotInUse = await helper.validateExistenceOfIds( 
         currAssocIds, currModel );
    let idsArePresent = ( idsNotInUse === 0 );
    if (!idsArePresent) {
      throw new Error(`ID ${idsNotInUse[0]} has no existing record in data model ${currModel.definition.model}`);
    }
    return acc && idsArePresent
  }, Promise.resolve( true ))
}

unique(inputArray) {
  return [...new Set(inputArray)]
}

sanitizeAssociationArguments( input, argNamesArray ) {
  // use above unique to return a copy of input
  // (make sure you copy and do not work on reference) - Object.assign..
  // all Array assoc inputs will be made unique
  return /* ... the unique-ified copy of input */
}

countRecordsInAssociationArgs( input, argNamesArray ) {
  ...
}

*Edit* (Thomas Voecking): The function `checkExistence` is now implemented via `await validateExistenceOfIds( ... ).length) === 0`. As a reference this is still here

// formerly known as 'checkExistence'
validateExistenceOfIds( idArr, model ) {

  // in SQL the following would be:
  // SELECT COUNT(${model.getIdAttribute()}) FROM ${model.tableName} WHERE ${model.getIdAttribute} in [${idArr.join(',')}]

  let searchArg = {search: field: model.getIdAttribute(), {operator: 'in', value: {value: idArr, type: "Array"}}
  
  try {
    // SINGLE statement that counts the persistent (existing) ids in argument 'idArr'
    let countIds = model.countRecords( searchArg )
    return countIds === idArr.length
  } catch err {
    if (/* err is not implemented */) {
      // do the same with "find by id" 
      // MULTIPLE statements are sent to the DB
      let allExist = idArr.reduce( function(acc, curr) {
        return acc && model.readById(curr) !== undefined
      }, true)
     
    } else throw err
  }
  
}

### A closed look at `handleAssociations`

// helper
isNonEmptyArray( a ) {
  return (a !== undefined && Array.isArray( a ) && a.length > 0)
}

isNotUndefinedAndNotNull( v ) {
  return (v !== undefined && v !== null)
}

#### Field resolvers

For each data model Cenzontle creates root and field resolvers. Field resolvers are functions available in the instances of records. In the following associations and their administration are handled as _instance functions_.

// field resolver for Person
// NOTE that 
// `this`
// points to an instance of the Person Data Model!

handleAssociations( input, context ) {
  // Do the following in parallel
  // The following associations are those 
  // where the foreign key is in the associated models:
  if ( isNonEmptyArray( input.addDogs ) ) {
    add_dogs( input, context ) }
  if ( isNonEmptyArray( input.removeDogs ) ) {
    remove_dogs( input, context ) }
  if (...) {
    addParrots( input, context ) }
  if (...) {
    remove_parrots( input, context ) }
  // The following associations are those
  // where the Foreign Key is inside the Person model:
  if ( isNotUndefinedAndNotNull(input.addEmployer) ) {
    add_employer( input, context ) }
  if (...) {
    remove_employer( input, context ) }
  // ...
}

// NOTE: 
// All add_<AssocName> and remove_<AssocName> should work in parallel

// Example implementation (assume foreign key in Parrot)
add_parrots( input, context ) {
  // extract and validate personId
  // use await Promise.all of the following:
  input.addParrots.forEach( parrotId =>
    // Example implementation below in 'addEmployer'
    models.Parrot._addPerson( parrotId, this.getIdValue() )
  ) 
}

remove_parrots( input, context ) {
  // extract and validate personId
  input.removeParrots.forEach( parrotId =>
    // Example implementation below in 'removeEmployer'
    models.Parrot._removePerson( parrotId, this.getIdValue() )
  ) 
}

// Example implementation (assume foreign key in Person)
async add_employer( input, context ) {
  // the data model layer implementation is now static:
  let result = await models.Person._addEmployer( this.getIdValue(), input.addEmployer )
  // no error then update the Person record
  this.employer_id = input.addEmployer
}

async remove_employer( input, context ) {
  // IMPORTANT: Do not forget this check!
  if (input.removeEmployer === this.employer_id) {
    let result = await models.Person._removeEmployer( this.getIdValue(), input.removeEmployer )
    this.employer_id = null
  }
}

### Model Layer Implementation for writing associations
Notes:
* These are the so called "underscore" implementation functions `_add<AssocName>` and `_remove<AssocName>`.
* These implementations only exists if the data model _holds_ the respective foreign-key.

// Parrot data model
// holds the foreign key 'person_id'
_addPerson(parrotId, personId) {
  // CONSIDER THE FOLLOWING possible CASES:
  // 1) local SQL
  // You can use the Sequelize 'setPerson' or execute the update SQL:
  `UPDATE parrots SET person_id = ${personId} WHERE id = ${parrotId}`
  // 2) generic web-service and remote cenzontle
  request.post( `mutation { updateParrot( parrotId: ${parrotId}, addPerson: ${personId}` )
  // 3) DDM
  let responsibleAdapter = /* forIri( parrotId ) */
  responsibleAdapter._addPerson( parrotId, personId )
}

_removePerson(parrotId, personId) {
  // CONSIDER THE FOLLOWING possible CASES:
  // 1) local SQL
  `UPDATE parrots SET person_id = NULL WHERE id = ${parrotId} AND person_id = ${personId}`
  // 2) generic web-service and remote cenzontle
  request.post( `mutation { updateParrot( parrotId: ${parrotId}, removePerson: ${personId}` )
  // 3) DDM
  let responsibleAdapter = /* forIri( parrotId ) */
  responsibleAdapter._removePerson( parrotId, personId )
}

### Considering the delete case

Basically we implement a restrictied deletion. A record can only be deleted if it has _no_ associated records, similar to the `ON DELETE RESTRICT` foreign-key constraint in relational databases.

// Person resolver
// ***************

deletePerson( {id}, context ) {
  // validation throws errors, if record is not valid
  validForDeletion( id, context )
  models.Person.deleteOne( id )
}

countAllAssociatedRecords( {id}, context ) {
  let person = readOnePerson( {id}, context )
  // do in parallel:
  // Promise.all
  person.countFilteredDogs( context )
  // NOTE
  // in case of associated DDMs
  // check "benign errors" for nodes that were not reachable (TimeOut or something similar)
  // In case such an error occurred, throw an Error and prevent from deletion!
  person.countFilteredParrots( context )
  person.employer( context ) !== undefined 
}

validForDeletion( id, context ) {
  // 1. count all associated records (-> assocTotalCount)
  let assocTotalCount = countAllAssociatedRecords( id, context )
  if (assocTotalCount > 0) {
    throw new Error(`<DataModelName> with <idAttributeName> ${id} has associated records and is NOT valid for deletion. Please clean up before you delete.`)
  }
  if (context.benignErrors.length > 0) {
    throw new Error('Errors occurred when counting associated records. No deletion permitted for reasons of security.')
  }
  return true
 

#### Special case `has_many_through_sql_cross_table`

NOTE: Check, if exhaustive definition?

Consider the User has many Role and Role has many User example. Cross table is `users_to_roles`.

// User resolver
deleteUser( ... ) { ... }

validForDeletion( id, context ) {
  // check cross model records
}

add_roles(input) {
  // analog to default add
}

// user model
_addRole( userId, roleId ) {
  // insert into cross-table
  // SOLUTION A)
  // Code Generator uses the "cross model"
  // i.e. the one labelled "keysIn" in the
  // association definition
  // be aware of the id type and transform accordingly if not String
  models.<keysIn>.addOne({userId: userId, roleId: roleId})
  // SOLUTION B)
  // Change our definition,
  // i.e. the domain specific language used to 
  // describe Cenzontle data models and their assocs.
  // Expect the 'crossTableName' to be present.
  `INSERT INTO <crossTableName> ("userId", "roleId", "createdAt", "updatedAt") VALUES ...`
 


_removeRole( userId, roleId ) {
  // delete from cross-table
  // analog to above 
}

##### Foreign-Key constraints

The code generator send the user a message and recommends to create a migration in which the user creates `ON DELETE RESTRICT` foreign-key constraints, which actually will be in accordance with the default Cenzontle behaviour.

### Generic Associations?

**Do we need generic associations? If and only if so, consider the following design sketches.**

How to handle associations of type generic_to_one or generic_to_many?

// Above example of Person and Dog, Parrot, and Employer
// Person resolver generic_to_one( Employer )
handleAssociations(...) { ... }
add_employer( input, context ) {
  // check authorization
  // - either on Model if not DDM
  // - or on Adapter if associated Model is a DDM
  // because, we do know nothing about the implementation
  // - i.e. if there is a foreign-key, and if so, where -
  // always invoke the dataModel._addEmployer
  models.Person._addEmployer( ... )
}
// person data model
_addEmployer( ... ) {
  /* YOUR CODE GOES HERE */
  throw new NotImplError
}

How to handle Data Models of storageType Note, we are considering the whole data model, not only the associations!

// person.json
{
  model: 'Person',
  attributes: {
    ...
  },
  storageType: 'generic'  // <<<<<<======
  associations: {
    dogs: {
     type: "generic_to_one" | "generic_to_many" // others are not allowed
     target: "Dog"
    },
    employer: {
      type: "generic_to_one",
      target: Employer
    }
   }
}
// person resolver
// ALL of the below resolvers check validation and
// if they are readers of many, do the countImpl (data model), check limit, fetchImpl (data model), check limit algorithm
// if not forward simply to the respective "Impl" in the data model
// IMPLEMENTS THE CENZONTLE API, i.e
countPeople(...){...}
peopleConnection(...){...}
peopleFilter(...){...}
readOnePerson(...){...}
addPerson(...){...}
updatePerson(...){...}
deletePerson(...){...}
// etc
// associations (see above about 'generic' associations
countFilteredDogs(...){...}
dogsFilter(...){...}
dogsConnection(...){...}
employer(...){...}
// person data model
// Has all the Impls that do nothing more than throw the not implemented error!

### What will change in comparison to the original plan?
* Writing associations will not be done in the create, update model functions, but will be delegated in the resolvers - see handleAssociations
* Data Models will only have _add<AssociatedModel> and _remove<AssociatedModel> if and only if the data model itself holds the foreign-key.
* Authorization checks when writing associations: We require read _and_ update permissions on associated data models, additionally to write permission on the invoked data model.
* Consistency insurance when deleting a record by only allowing deletion of records that have no associated records - similar to RDB's ON DELETE RESTRICT.

### Can be made more efficient

The underscore functions _addEmployer or _addPerson, now that they are static can also be made to accept an Array of foreignKeys.
Thus addParrots would not need to iterate over all parrotIds, but a single write operation can be executed.
We would need to change the schema for this, so that it works with remote Data Models.

#### Implementation sketch

Offer bulk association that can be invoked as a root resolver.

##### GraphQL-schema

Note, that the input generated in the GraphQL-Schema has two arguments, each of which identifies the `ID`s at one end of the association that is to be persisted as bulk. The type of the respective arguments depends on whether the end in question is a `to-one` or a `to-many` end. In the former case a single `ID!` is required, in the latter an Array of `[ID!]!` is required. Any type of associations, `to_one`, `to_many`, and `many_to_many` (to be implemented in the future) can thus be set without ambiguity.

""" Models: Person, Dog, Parrot, Employer (see above examples) """

""" Example for a to_many association """
""" The Person will be associated with all Dogs in the following input """
input bulkAssociatePersonWithDogInput {
  personId: ID!
  dogIds: [ID!]!
}

""" Example for a to_one association """
""" The Luke will be associated with the Vader identified by their respective IDs """
input bulkAssociateLukeWithDarthVaderInput {
  lukeId: ID!
  iAmYourfatherId: ID!
}

""" Example for a many_to_many association """
""" All roles will be associated with all users in this input argument """
input bulkAssociateUsersToRolesInput {
  usersId: [ID!]!
  rolesIds: [ID!]!
}


""" Note, that you can provide an ARRAY of inputs - the actual bulk"""
bulkAssociatePersonWithDog([bulkAssociatePersonWithDogInput])

##### UPDATE July 06 2020
For now only implement the second schema (bulkAssociateLukeWithDarthVaderInput) only in data model that hold the foreign key. The user will be expected to input the associations one by one, even if it is a to_many case (Person has many dogs). For example:

[{personId: 1, dogId: 2}, {personId: 1, dogId: 3}, {personId:1, dogId:4}, ...]

The below design of the model function still holds true of course. The actual database operation should be as efficient as possible.

In the Future we might offer a richer API for user input like this:

[{personId:1, dogIds: [2,3,4]}]

This can be done by either adding a "other_end" field in the JSON definition or by going back to naming the whole association eg. _one_to_many_, _many_to_one_, etc..

##### Resolver and Model bulkAssociate

The above schema will be implemented in each Data Model for each of its associations. Note, that for a single of the above `bulkAssociatePersonWithDogInput` in most storages we can create and execute a single write operation.

// parrot data model
_bulkAssociatePersonWithParrot([{personId, parrotIdsArray}]) {
  // deal with Array input accordingly
  // assume SQL storage type
  `UPDATE parrots SET person_id = ${personId} WHERE parrots.id in [${parrotIdsArray.join(",")}]`
  // generic web-service and remote cenzontle server
  // invoke the new root resolver `bulkAssociatePersonWithDogInput`
  axios.post( `mutation bulkAssociatePersonWithDogInput(...) {...}`)
  // In the case of DDMs:
  // Basically split argument parrotIdArray into chunks for each responsible adapter and 
  // send the mutation requests to each of those:
  let parrotIdsMappedToResponsibleAdapters = helper.mapIdsToResponsibleAdapters(...)
  // above returns { 'adapterNameOne': /*subset of parrotIdsArray */, 'adapterNameTwo': /* ... */}
  // iterate over Object.keys() and send 
}

##### Usage in add<AssocName> and remove<AssocName> resolver-functions

Consider the above example

// Example implementation (assume foreign key in Parrot)
add_parrots( input, context ) {
  if (/* valid input */) {
    // INEFFICIENT current implementation, because |input.addParrots|
    // write operations will be executed.
    // extract and validate personId
    input.addParrots.forEach( parrotId =>
      models.Parrot._addPerson( parrotId, input.personId )
    ) 
    
    // EFFICIENT single write operation implementation
    // Because the foreign key is in Parrot,
    // we still invoke the underscore implementation in the Parrot Data Model:
    models.Parrot._bulkAssociatePersonWithParrot(personId, input.addParrots)
}

##### UPDATE July 15 2020
* The remove case can be handeled analog with a `bulkDisAssociate` function.
* use the skipAssociationsExistenceChecks Argument to skip the superflous existence checks for ddm-adapter/ zendro-adapter / zendro-webservice.

## List of functions to be implemented

Functions are sorted by call-complexity (CC), i.e. how many other functions invoke them.
List is sorted from most complex to least.

We found that they can be separated into two independent sets.

#### ToDos

- remove add<Assoc> and remove<Assoc> from the GraphQL query send to remote servers in the case of a remote adapters and remote storage types, i.e. all except SQL type.
2.- _add<Association>, _remove<Association> model layer - Constantin
- remote web-service (cenzontle)
- generic web-service
- DDM
- adapters
3.- add<Association>, remove<Association> resolver - Constantin, Thomas
4.- add<Association>, remove<Association> DDM resolver - Constantin, Thomas
- removeAssociationArgs in add and update resolvers
- cross table special case

##### DONE

7.- add<ModelName> resolver - Constantin
7.- update<ModelName> resolver - Constanin

5.- handleAssociations( ... ) - Constantin

1.- countAllAssociatedRecords( ... )
2.- validateForDeletion( ... )
3.- delete( ... )

2.- checkAndAdjustRecordLimitForCreateUpdate( ... ) - Francisco
1.- checkExistence( ... ) - Francisco, Thomas
1.- validateExistenceOfIds(...) - former method `checkExistence`
1.- checkAuthorizationOnAssocArgs(...) - Thomas
1.- associationArgsDef(...) - declare a `const` in the resolver
1.- countRecordsInAssociationArgs( ... ) (already done by Thomas)
1.- isNonEmptyArray( ... ) ( already done by Thomas)
1.- isNotUndefinedAndNotNull( ... ) (already done by Thomas)
1.- sanitizeAssociationArguments(...)
2.- validateAssociationArgsExistence( ... ) - Francisco, Thomas