# TODOs - Project Walkthrough
---
# User story
* users can
* sign up
* get activation token from email
* login using email, password
* access list of tasks own by themself
* Do CRUD on tasks
----
## API Design
----
### Healthcheck
* `/v1/healthcheck` :
* `GET`: Show status of service.
----
### tasks
* `/v1/tasks`:
* `GET`: get all tasks for specific user.
* `POST`: Create a new task for specific user.
* `/v1/tasks/{taskID}`:
* `GET`: Get task by ID for specific user.
* `DELETE`: Delete task for specific user.
* `PATCH`: Update task for specific user.
----
### users
* `/v1/users/activation`
* `PUT`: Activate user based on given token.
* `/v1/users/registration`:
* `POST`: Register user based on given information.
----
### tokens
* `/v1/tokens/authentication`:
* `POST`: Create authentication token for user.
---
# Clean Architecture
- :question: Why I use clean architecture ?
- easy to test
- integration test, unit test
- same business logic, different delivery method
- e.g. RESTful API, gRPC
- :question: How to seperate each layer ?
- By interface!
----
## Overview
<img src="https://i.imgur.com/YfbePsH.png"></img>
[link](https://gitmind.com/app/flowchart/c976955010)
----
## Domain entities
----
### Token
:question: How token is generated ?
1. Put randomBytes in 16-byte-loing slice from operating system's CSPRNG.
2. Encode the bytes to base-32-encoded string, and store it in `token.Plaintext`
3. For `token.Hash`, use `sha256` hash function
----
### User
* Only store hash in database
* use bcrypt to check if password match.
----
### Task
* title
* content
* done
* timestamp
----
### Database Schema
* token.user_id is the foreign key which points to user.id
* task.user_id is the foreign key which points to user.id
----
```mermaid
erDiagram
user {
id bigserial
created_at timestamp
name text
email citext
password_hash bytea
activated bool
version integer
}
token {
hash bytea
user_id bigint
expiry timestamp
scope text
}
task {
id bigserial
user_id bigint
created_at timestamp
title text
content text
done boolean
version integer
}
user ||..o{ token: has
user ||..o{ task: has
```
----
## Delivery
* tested by mocking usecase layer (using [`stretchr/testify`](https://github.com/stretchr/testify))
* set expectations
* inject test data to delivery layer
* assert outputs are what we expected.
* assert expectations are met.
----
## Usecase
* contains some business logic
* tested by mocking repository layer using [`stretchr/testify`](https://github.com/stretchr/testify)
* e.g. If we want to test token usecase:
* we set up mock token repository
* set expectation
* assert expectation is satisfied.
* assert output of tokenUsecase is correct.
----
## Repository
* the code that make database connection to database
* tested by `testcontainers`
* Real database, not just test correctness of SQL query.
---
# Makefile
* include `.envrc` to set up
* POSTGRES_USER
* POSTGRES_PASSWORD
* POSTGRES_DB
* TODOS_APP_DB_DSN ...
* Makefile command tree
* we can use tab to do auto-complete
* e.g. Type `$ make run/ + [tab]`
* We get
* `run/api`
* `run/compose/down`
* `run/compose/up`
----
## Graph
```mermaid
stateDiagram-v2
make --> build/
make --> run/
make --> db/
make --> docs/
make --> audit
make --> vendor
build/ --> api
build/ --> docker/image(deprecated)
run/ --> api
run/ --> compose/
compose/ --> up
compose/ --> down
db/ --> migrations/
db/ --> connect
db/ --> start
db/ --> stop
migrations/ --> up
migrations/ --> down
docs/ --> gen
docs/ --> show
```
---
# Error Handling
* :question: Why I use custom error type ?
* We can define informative error message by just filling in the fields in `errors.Error`
* Integrate stacktrace mechanism of `pkg/errors`, and `ErrorStackMarshaler` in `rs/zerolog`
----
## Example: Error across different layer
```
(delivery) (usecase) (repository)
getHandler -> businessLogic -> getFromDB -> sql.ErrNoRows
```
----
## The error message we want
```
alice@example.com: getHandler: >> businessLogic: counter value = 3: >> getFromDB: kind database error: >> sql: no rows in result set
```
----
## Use `%+v` in formatted output
```
alice@example.com: getHandler: >> businessLogic: counter value = 3: >> getFromDB: kind database error: >> sql: no rows in result set
github.com/unknowntpo/todos/internal/domain/errors.E
/Users/unknowntpo/repo/unknowntpo/todos/feat-error/internal/domain/errors/errors.go:232
github.com/unknowntpo/todos/internal/domain/errors.getFromDB
/Users/unknowntpo/repo/unknowntpo/todos/feat-error/internal/domain/errors/example_test.go:10
github.com/unknowntpo/todos/internal/domain/errors.businessLogic
/Users/unknowntpo/repo/unknowntpo/todos/feat-error/internal/domain/errors/example_test.go:16
github.com/unknowntpo/todos/internal/domain/errors.getHandler
/Users/unknowntpo/repo/unknowntpo/todos/feat-error/internal/domain/errors/example_test.go:26
github.com/unknowntpo/todos/internal/domain/errors.Example
...
```
----
## Full code
----
### Repository layer
```
func getFromDB() error {
const op Op = "getFromDB"
return E(op, KindDatabase, sql.ErrNoRows)
}
```
----
### Usecase layer
```
func businessLogic() error {
const op Op = "businessLogic"
cnt := 3
err := getFromDB()
if err != nil {
return E(op, Msg("counter value = %d").Format(cnt), err)
}
return nil
}
```
----
### Delivery layer
```
func getHandler() error {
const op Op = "getHandler"
const email UserEmail = "alice@example.com"
err := businessLogic()
if err != nil {
// Handle error here
switch {
case KindIs(err, KindDatabase):
return E(op, email, err)
default:
// Do something else
}
}
return nil
}
```
----
## Integration with zerolog
* use `panic()` in user repository as example
* The log shows the stack trace of where panic happends
* Assume we have panic at `(*userRepo).GetByEmail`
```
{
"level": "error",
"stack": [
{
"func": "New",
"line": "174",
"source": "errors.go"
},
{
"func": "(*Middleware).RecoverPanic.func1.1",
"line": "39",
"source": "middleware.go"
},
{
"func": "gopanic",
"line": "965",
"source": "panic.go"
},
{
"func": "(*userRepo).GetByEmail",
"line": "56",
"source": "postgres_repo.go"
},
{
"func": "(*userUsecase).Login",
"line": "94",
"source": "user_usecase.go"
},
{
"func": "(*tokenAPI).CreateAuthenticationToken",
"line": "64",
"source": "token_api.go"
},
{
"func": "HandlerFunc.ServeHTTP",
"line": "2050",
"source": "server.go"
},
{
"func": "(*Router).Handler.func1",
"line": "275",
"source": "router.go"
},
{
"func": "(*Router).ServeHTTP",
"line": "387",
"source": "router.go"
},
{
"func": "(*Middleware).Authenticate.func1",
"line": "123",
"source": "middleware.go"
},
{
"func": "HandlerFunc.ServeHTTP",
"line": "2050",
"source": "server.go"
},
{
"func": "(*Middleware).RateLimit.func2",
"line": "108",
"source": "middleware.go"
},
{
"func": "HandlerFunc.ServeHTTP",
"line": "2050",
"source": "server.go"
},
{
"func": "(*Middleware).EnableCORS.func1",
"line": "262",
"source": "middleware.go"
},
{
"func": "HandlerFunc.ServeHTTP",
"line": "2050",
"source": "server.go"
},
{
"func": "(*Middleware).RecoverPanic.func1",
"line": "43",
"source": "middleware.go"
},
{
"func": "HandlerFunc.ServeHTTP",
"line": "2050",
"source": "server.go"
},
{
"func": "serverHandler.ServeHTTP",
"line": "2868",
"source": "server.go"
},
{
"func": "(*conn).serve",
"line": "1933",
"source": "server.go"
},
{
"func": "goexit",
"line": "1371",
"source": "asm_amd64.s"
}
],
"error": "something wrong in userRepo",
"time": "2021-12-14T01:30:58Z"
}
```
---
# Dockerfile
----
## Multi-stage build
```mermaid
graph TD
A(config-base) -->
| copy Makefile<br/>.envrc<br/>golang-migrate binary file<br/>migration files<br/>testdata<br/>config.sh|B(config)
C(build-base<br/><build the binary file>) -->
|copy binary file<br/>app_config-prod.yml| F(production)
E(scratch) -->|as base image| F(production)
```
* Parallel build
* When we change content in build base, config won't be changed.
* `.envrc`
---
# Configuration management
Use [spf13/viper](https://github.com/spf13/viper) for configuration management
* :question: The order of config variable parsing ?
* in `go doc viper.Get`
* override, flag, env, config file, key/value store, default
* We use env, config file, default only
----
## Order of injecting configuration value:
1. env, TODOS_*
2. config file *.yml
3. default config in cmd/api/config.go
---
# Graceful shutdown
* app.serve() starts a new background goroutine `go1`
* `go1` monitor signal `syscall.SIGINT` `syscall.SIGTERM`
* If `go1` got signal
* shutdown the worker pool
* shutdown the server by `http.Server.Shutdown()`
* send the error occured during server shutdown to a channel `shutdownErr`
* wait for the pool to stop
----
## In background monitoring goroutine
```go
go func() {
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
s := <-quit
// Shutdown worker pool.
poolCancel()
app.logger.PrintInfo("shutting down server", map[string]interface{}{
"signal": s.String(),
})
// do server shutdown routine.
serverCtx, serverCancel := context.WithTimeout(context.Background(), 3*time.Second)
defer serverCancel()
err := srv.Shutdown(serverCtx)
if err != nil {
shutdownErr <- err
}
app.pool.Wait()
shutdownErr <- nil
}()
```
----
## In `app.serve()`
```go
func (app *application) serve() error {
...
app.logger.PrintInfo("starting server", map[string]interface{}{
"addr": srv.Addr,
"env": app.config.Env,
"version": version,
})
err := srv.ListenAndServe()
if err != nil {
if !errors.Is(err, http.ErrServerClosed) {
// Don't need to do graceful shutdown process, just return the error.
return err
}
}
err = <-shutdownErr
if err != nil {
return err
}
app.logger.PrintInfo("stopped server", map[string]interface{}{
"addr": srv.Addr,
})
return nil
}
```
---
# Drone CI/CD
Three steps:
1. Local
2. Test on Drone server
3. Deploy
----
## 1. Local:
* pre-commit hook
* Check mockery
* Check make vendor
* make audit (do some test)
* Check swagger
----
## 2. Test on Drone server
* unit test
* integration test
* not finished, due to docker-in-docker storage driver problem
* push to docker hub
----
## 3. Deploy
* Use `drone-ssh` plugin
* pull image from docker hub
* deploy
* drone secret set environment variables in `.envrc`
---
# Improvements
* Write integration tests for mailstrap smtp server
* Add new feature: Reminder
* background goroutine
* get batch of tasks to be reminded
* send reminder message through *LINE Notify*
* Oauth: Github, Google, ...
---
# Thank You
{"metaMigratedAt":"2023-06-16T14:46:41.173Z","metaMigratedFrom":"Content","title":"TODOs - Project Walkthrough","breaks":true,"description":"users can","contributors":"[{\"id\":\"7bff8b8c-2096-4393-908e-9d3f7b516e17\",\"add\":23474,\"del\":11825}]"}