# Databless
## What we love in Databless
❤️ We love using it, various projects
❤️ Adapted easily
[Databless readme on GitHub](https://github.com/AckeeCZ/databless/tree/next)
### Model
- not defined attributes are filtered before inserting/updating
```javascript=
export interface SubscriptionPlan {
id: string
type: SubscriptionPlanType
price: number
dishesCount: number
name: string
}
export const subscriptionPlanModel = createModel<SubscriptionPlan>({
adapter: getKnex,
collectionName: 'subscription_plans',
attributes: {
id: { type: 'string' },
price: {
type: 'number',
deserialize: value => (isNaN(Number(value)) ? null : Number(value)),
},
type: { type: 'string' },
dishesCount: { type: 'number' },
name: { type: 'string' },
},
})
```
- custom serialization
```javascript=
objectStoredAsJson: {
type: 'string',
// serialize before inserting into a database
serialize: x => JSON.stringify(x || null),
// deserialize from database shape
// deserialize can also be used to define attribute TS type, e.g. (x: string): MyEnum => x
deserialize: x => JSON.parse(x),
}
```
### Repository
- CRUD methods
- `.create(data, options)`
- `.createBulk(data[], options)`
- `.detail(filters, options)`
- `.delete(filters, options)`
- `.list(filters, options)`
- `.update(filters, data, options)`
```javascript=
export const subscriptionPlanRepository = createRepository(
subscriptionPlanModel
)
```
### Filtering
- via `filters`
- exact match
- `{ foo: 'bar' }`
- where in
- `{ foo: ['bar', 'baz'] }`
- inequality (prefix `>`, `<`, `>=`, `<=`)
- `{ foo: '>2' }`
- searching (left & right wildcards)
- `{ foo: '*abc*' }`
- `{ foo: 'abc*' }`
- `{ foo: '*abc' }`
- custom queries
- via `options.qb` parameter
- be careful of SQL query
- `{ qb: (qb: Knex.QueryBuilder) => qb.whereRaw('...') }`
- custom model filters
- do not overwrite default Model attribute filters
- `addressRepository.list({ userId })`
```javascript=
export const addressModel = createModel<
Address,
{ customFilters: { userId: string } }
>({
adapter: getKnex,
collectionName: 'addresses',
attributes: {
id: { type: 'string' },
name: { type: 'string' },
street: { type: 'string' },
zip: { type: 'string' },
city: { type: 'string' },
countryCode: { type: 'string' },
note: { type: 'string' },
},
filters: {
userId: (value: string, options) => {
if (!value) {
return
}
options.qb = composeQb(options.qb, qb => {
return qb
.join('user_addresses', 'addresses.id', 'user_addresses.address_id')
.where('user_addresses.user_id', value)
.orderBy('user_addresses.is_default', 'desc')
})
},
},
})
```
### Counting
- via `options.count`
- `count: true` to get number of filtered records
### Ordering
- via `options.order`
- default
- `{ order: 'foo' }`
- asc
- `{ order: '+foo' }`
- desc
- `{ order: '-foo' }`
- order by multiple
- `{ order: ['+foo', '-baz'] }`
### Pagination
- via `options.limit` and `options.offset`
- defaults: `limit=10, offset=0`
- `{ limit: 10, offset: 0 }`
### Relations
- attribute type of `relation`
- example `AccountGroup` model
```javascript=
accounts: {
type: 'relation',
targetModel: () => accountModel,
relation: bookshelfRelation.createHasMany({
foreignKey: 'accountGroupId'
})
},
```
- related properties populated on `list` and `detail` repository methods within `withRelated`
```javascript=
const book = bookRepository.detail({ id }, { withRelated: ['author' ] })
```
## What we hate in Databless
- ❗ `createBulk`
- does not return all items inserted
```javascript=
[
{
command: 'INSERT',
rowCount: 2,
oid: 0,
rows: [],
fields: [],
parsers: undefined,
types: TypeOverrides { _types: [Object], text: {}, binary: {} },
rowCtor: null,
rowAsArray: false
}
]
```
```javascript=
// workaround for getting entities you created (or you have to list)
const [firstChef, secondChef] = await Promise.all([
userRepository.create({
fullName: 'test_chef_1',
uid: 'test_chef_1',
}),
userRepository.create({
fullName: 'test_chef_2',
uid: 'test_chef_2',
}),
])
```
- ❗ `delete`
- do not know whether delete operation happened or not
- ❗ `detail`
- when `undefined` filter argument passed it returns the first item from a table
- IRL example
```javascript=
facebook: {
getUserByEmail: email => getUser({ email }),
},
....
const getUser = async (
params: Partial<AuthistUser> & { id?: string }
): Promise<AuthistUser | undefined> => {
const user = await userRepository.detail(params, {
withRelated: ['roles', 'settings']
})
if (!user) {
return
}
....
```
- ❗ `detail`
- does not have return type of `underfined` or `null`
- example with empty table `const x = await repository.detail({ id: '1' })`
- ❗ `list`
- when `undefined` filter argument passed it does returns all items
- ❗ `update`
- if item to be updated not found, it returns object with data to be updated
- ❗ `camelCase` sometimes does not work
- Model with `users` relation attribute example
```javascript=
users: {
type: 'relation',
targetModel: () => userModel,
relation: bookshelfRelation.createBelongsToMany({
joinTableName: 'subscriptionAddonsUsers',
otherKey: 'userId',
foreignKey: 'subscription_addon_id'
})
}
```
- ❗ Missing properties on `AttributeRelation` type
- such as `serialize / deserialize` properties
- example `User` model with relation attribute returns TS error
```javascript=
roles: {
type: 'relation',
targetModel: () => roleModel,
relation: bookshelfRelation.createHasMany({
foreignKey: 'userId'
}),
// @ts-expect-error remove after databless fix - https://github.com/AckeeCZ/databless/issues/24
deserialize: (roles?: Array<{ role: string }>) =>
(roles ?? []).map(role => role.role)
}
```
- ❗ Typescript support for nested queries within `withRelated`
- example
```javascript=
const users = await userRepository.list({}, {
withRelated: [{
'roles': function (qb: Knex.QueryBuilder) {
qb.where('role', 'user');
}
}] // as any
})
```
```
error TS2322: Type '{ roles: (qb: QueryBuilder<any, any>) => void; }' is not assignable to type '"accountGroups" | "roles" | "settings"'.
Type '{ roles: (qb: QueryBuilder<any, any>) => void; }' is not assignable to type '"settings"'.
```
- ❗ Typescript support for nested relations
- `withRelated: ['order', 'destination', 'order.partner' as any]`
- ❗ Typescript support for transactions in `RepositoryMethodOptions`
```javascript=
const address = await addressRepository.create(
{
street: input.street,
name: input.displayName,
},
{ transacting: tx } // have to type cast to any
)
```
```
error TS2345: Argument of type '{ transacting: Knex.Transaction<any, any[]>; }' is not assignable to parameter of type 'RepositoryMethodOptions'.
Object literal may only specify known properties, and 'transacting' does not exist in type 'RepositoryMethodOptions'.
```
- ❗ Redundant model interface
- we define `attributes` types and also almost the same in Model `interface`
- ❗ Circural reference
- example on FlashNews
- `dataChannel` entity has relation to `source` entity
```javascript=
// dataChannel model
source: {
type: 'relation',
targetModel: () => sourceModel,
relation: bookshelfRelation.createBelongsTo({
foreignKey: 'sourceId',
}),
},
// source model
producingDataChannels: {
type: 'relation',
targetModel: () => getDataChannelModel(),
relation: bookshelfRelation.createHasMany({
foreignKey: 'sourceId',
query: collection =>
collection.query(qb => {
void qb
.where('dataChannel.state', DataChannelState.Active)
.andWhere('dataChannel.hideContent', false)
}),
}),
},
// getter function for dataChannel model
export const getDataChannelModel = (): Model<any> => dataChannelModel
```
## What we should improve in Databless
- General improvements to types. Some examples:
- Correct return type for repository methods
```javascript
const detail = async <E extends Entity, M extends Metadata<E>>(
model: Model<E, M>,
filter?: Filters<E, M>,
options?: RepositoryDetailOptions<E, M>
): Promise<E | undefined> => {...};
const remove = async <E extends Entity, M extends Metadata<E>>(
model: Model<E, M>,
filter?: Filters<E, M>,
options?: RepositoryMethodOptions): Promise<E | undefined> {...}
```
- Fix Type support for `columns` option when fetching the entity
```typescript
export interface RepositoryMethodOptions {
toJSON?: bookshelfUtil.SerializeOptions;
qb?: bookshelfUtil.QbOption;
columns?: string[];
}
```
- Type support for `transacting` option
```typescript
export interface RepositoryMethodOptions {
...,
transacting: any
}
```
- Type for serialize and deserialize on Attribute relation
```typescript
type AttributeRelation<R extends Relation = any> = {
type: 'relation';
...
serialize?: (x: any) => PrimitiveToType<Primitive>;
deserialize?: (x: any) => any;
};
```
- Correct types for "nested" `withRelated` options
- Create bulk: correct return value/type
```typescript
export const createBulk = async <E extends Entity, M extends Metadata<E>>(
model: Model<E, M>,
data: Array<Partial<NullishOptional<E>>>,
options?: RepositoryMethodOptions
): Promise<E[]> => {
const result = await model.options
.adapter()
.batchInsert(
model.options.collectionName,
data.map(dataItem => model.serialize(dataItem))
// TODO Support transactions using .transacting(trx)
// TODO Add timestamps if bsModel.hasTimestamps
)
.returning(options?.columns ?? '*');
return result as E[];
};
```
- Proper return value/type for the delete function. In case we need to know if delete did happen or not
- Update operation should return undefined if entity was not found
- Camel casing - We should have it documented: where it does and does not work.
- Defining models once instead of creating an Interface + Bookshelf model. As a example from other ORMs, we could define a single class to manage the model
```typescript=
// objection.js
export default class Animal extends Model {
id!: number
name!: string
owner?: Person
static tableName = 'animals'
static jsonSchema = {
type: 'object',
required: ['name'],
properties: {
id: { type: 'integer' },
ownerId: { type: ['integer', 'null'] },
name: { type: 'string', minLength: 1, maxLength: 255 },
species: { type: 'string', minLength: 1, maxLength: 255 },
},
}
static relationMappings = () => ({
owner: {
relation: Model.BelongsToOneRelation,
// The related model.
modelClass: Person,
join: {
from: 'animals.ownerId',
to: 'persons.id',
},
},
})
}
// MikroORM
@Entity({tableName: 'books'})
export class Book extends CustomBaseEntity {
@Property()
title!: string;
@ManyToOne(() => Author)
author!: Author;
@ManyToMany({ entity: 'BookTag', fixedOrder: true })
tags = new Collection<BookTag>(this);
}
```
## What we can add to Databless
New repository methods:
- Create a new instance without saving it to DB
```typescript
create2(entityLike): Entity
```
- Upsert
```typescript
upsert(entityLike: Partial<Entity> | Partial<Entity>[], options) Promise<Entity>
```
- Find and count
```typescript
findAndCount(filters, options): Promise<[Entity[], count]>
```
- Soft delete and restore
```typescript
softDelete(id, criteria): Promise<Entity>
restore(id, criteria): Promise<Entity>
```
New features:
- Cursor pagination support. We could use Relay cursor specification as a standard
- Caching support via options parameter (memory or redis)
```typescript
cache: {
type: "redis" | "memory",
options: {
host: "localhost",
port: 6379
}
}
```
- Locking support via options parameter
```typescript
{ lockMode: LockMode.OPTIMISTIC, lockVersion: req.body.version }
{ lockMode: LockMode.PESSIMISTIC_WRITE,}
```
- Virtual properties
- Lazy scalar properties
```typescript
text: { type: 'string', lazy: true },
```
- Events and Lifecycle Hooks
```typescript
const User = model('User', {
tableName: 'users',
initialize() {
this.on('updated', (model) => {
// This is fired after a model is updated
})
}
})
```
- Dataloader support for the relations
## Should we continue using Databless?
{"metaMigratedAt":"2023-06-16T20:45:01.034Z","metaMigratedFrom":"Content","title":"Databless","breaks":true,"contributors":"[{\"id\":\"c7b828ae-3e58-405d-9071-69503c09eb7f\",\"add\":9003,\"del\":376},{\"id\":\"6687b6a2-771a-4fd8-9f5d-120ab9b2f11a\",\"add\":15599,\"del\":10703}]"}