gc swiss army knife roundup
===========================
# maart 2021: cli brainstorm v1
https://hackmd.io/HlsWcOpoRLyx-mtIcLu1PQ
# leon: nodejs cli mockup v1
lightweight, spec-based, multi-command nodejs:
```
$ cat src/cmd/helloworld.js
module.exports = (gctl) => {
gctl.helloworld = async (opts) => {
console.log("helloworld! "+opts.value)
opts.greetings = opts.greetings || []
opts.greetings.push( new Date() )
// advanced: (recursively) call or chain (other) cmds (SUPER commands e.g.)
if( String(opts.value).match(/recursive/) ) return
await gctl.dry({value:123}) // call other cmd directly
gctl.execute({ // call other subcmd
_order:[
{key:"helloworld", value:"recursive1"},
{key:"helloworld", value:"recursive2"}
]
})
}
gctl.cmds.push({ // automatic help generation
names: ['helloworld', 'hw'],
type: 'string',
help: 'simple example cmd',
//helpArg: 'HELLOWORLD'
})
$ ./gctl
usage: ./gctl [OPTIONS]
options:
-c ARG, --collections=ARG commaseparated, or use * to select all
collections.
--ns=ARG, --nostate=ARG exclude _state$ or runtime collections
(--project enforces this).
--dry=ARG dry run (dont execute anything).
-e ARG, --exec=ARG execute cmd on selected host.
--hw=ARG, --helloworld=ARG simple example cmd.
-h ARG, --host=ARG select a host: gcaldi, gcv1, gcv2.
-v, --verbose Verbose output. Use multiple times for more
verbose.
```
# joost: spec mockup v1.1
https://hackmd.io/Wv7ZaxgbTl6lFit74SsUNA
> lange termijn: remote controllable stack dmv. websocketd
# leon: bash cli mockup v1.1
[Deze bash](https://gitlab.com/groundcontrol-app/gctl/-/blob/master/gc) versie
is geinspireerd op laatste spec, met als verschil:
* selectors (`gc:live` e.g.) vs actions (`db`)
* multi-actions voor zowel bash EN nodejs
* server/monitoring tools (`stack_init` om te installeren)
```
$ gc
usage: ./gc <cmd> [cmd] ...
CMDS:
stack_init install tcptrack && ss (tiny) monitor utils
servers
loadavg view server load (selectors: server)
sockets get sockets statistics
tcptrack realtime monitor server connections
db copy database (selectors: from, to)
SELECTORS:
server=uri gc:<stack>/<env>, mongodb:<uri>, gc:int, gc:acc, or gc:live
from=uri gc:<stack>/<env>, mongodb:<uri>, gc:int, gc:acc, or gc:live
to=uri gc:<stack>/<env>, mongodb:<uri>, gc:int, gc:acc, or gc:live
SERVERS:
- gc:local
- gc:int
- gc:live
- gc:aldi
- gc:v1
examples:
./gc server=gc:aldi loadavg server=gc:v2 loadavg
./gc from=gc:aldi to=gc:local db
```
> NOTE: dit startpunt heeft mn voorkeur met evt. nodejs subcommando's voor mongodb plumbing.
# leon: bash cli v1.2
Proberen:
```
intikken in je browser en opslaan:
https://gitlab.com/groundcontrol-app/gctl/-/raw/master/bash/gc
$ chmod 755 gc
$ ./gc
usage: ./gc <cmd> [cmd] ...
examples:
./gc int_to_acc src_sync
./gc acc_to_live src_sync
./gc server=gc:aldi loadavg server=gc:live loadavg
CMDS:
src_sync rsync stack-environment (selectors: from, to)
alias <name> <cmd ...> create alias for oneliners (edit ~/.gc for multiliners)
install_stack ubuntu: install docker + nginx-proxy-manager + acc/live env's
scale <up|down> docker: scale api/sender between 1/1 or 7/4 (using docker-compose)
db_sync copy database (selectors: from, to, db_strategy, collections)
db_collections_list list mongo collection (selectors: PORT_LOCAL)
db_dump list mongo collection (selectors: LOCAL_PORT)
install_utils install tcptrack && ss (tiny) monitor utils
loadavg print server load (selectors: server)
sockets print sockets statistics
tcptrack realtime monitor server connections
ping ping all related URLS of selected server
servers list all servers (from ~/.ssh/config)
sshtunnel open sshtunnel (selectors: server, port_local, port_remote)
ALIASES: stored in /home/leon/.gc
health
foo
SELECTORS:
server=uri gc:<stack>/<env>, mongodb:<uri>, gc:int, gc:acc, or gc:live
from=uri gc:<stack>/<env>, mongodb:<uri>, gc:int, gc:acc, or gc:live
to=uri gc:<stack>/<env>, mongodb:<uri>, gc:int, gc:acc, or gc:live
int_to_acc alias for: from=gc:int to=gc:local/acc
acc_to_live alias for: from=gc:local/acc to=gc:/local/live
interactive interactive=1 (default), use interactive=0 for scripts
port_local=<port> port for sshtunnel/database use
port_remote=<port> port for sshtunnel/database use
user <username> username to use with sshtunnel/database
db_strategy=<strategy> database sync strategy ("merge" or "drop_and_import")
collections=<foo bar> select mongo collections (space-separated, * = all)
SERVERS: <env> can be: acc, live, int
- gc:local
- gc:int
- gc:live
- gc:aldi
- gc:v1
```
De `./gc int_to_acc src_sync` werkt al (equivalent van ./sync), source is [hier](https://gitlab.com/groundcontrol-app/gctl/-/blob/master/bash/cmd/sync)
## Stack-like execution
Dit was nodig aangezien de spec (https://hackmd.io/Wv7ZaxgbTl6lFit74SsUNA) een beetje dubbelzinnig werd kwa herbruikbare cmds.
makkelijker format: `./gc <selector> <cmd> [...]` e.g.
```
$ ./gc server=gc:aldi loadavg server=gc:live loadavg
[gcaldi] [loadavg]
[gcaldi] 0.16 0.09 0.08 2/1324 16022 diskspace: 34% MemFree: 4188200 kB
[gclive] 0.14 0.10 0.09 1/1265 4181798 diskspace: 17% MemFree: 1322396 kB
```
Dit kan je lezen als: `loadavg(server) ; loadavg(server)`.
Heel simpel, gewoon global vars en functies.
## Alias
gooi gewoon dit in `~/.gc`:
```
pingall(){
_gc server=gc:aldi ping server=gc:live ping
}
```
> en dan kan je gewoon runnen:
```
$ ./gc pingall
[gcaldi] [ping]
[gcaldi] [ONLINE] mission.aldi.groundcontrol.app/login
[gcaldi] [ONLINE] api.aldi.groundcontrol.app
[gcaldi] [ONLINE] mission.acc.aldi.groundcontrol.app/login
[gcaldi] [ONLINE] api.acc.aldi.groundcontrol.app
[gclive] [ONLINE] mission.groundcontrol.app/login
[gclive] [ONLINE] api.groundcontrol.app
[gclive] [ONLINE] mission.acc.groundcontrol.app/login
[gclive] [ONLINE] api.acc.groundcontrol.app
```
> `./gc alias foo bar` subcommando werkt half, en moet nog ff de stackbased parser disablen (die voert `foo bar` ook uit haha).
## prompt missing selectors/args
missende selectors word zo in een commando opgevangen:
```
sshtunnel(){
_ensure_selector PORT_LOCAL
_ensure_selector PORT_REMOTE
...
```
demo:
```
$ ./gc server=gc:int sshtunnel loadavg
[gcint] [sshtunnel]
[missing selector: port_local] enter value: 8888
[missing selector: port_remote] enter value: 27017
[gcint] closing (previous) sshtunnels..
[gcint] 17780
[gcint] $ ssh groundcontrol@gcint -fN -L 8888:127.0.0.1:27017
[gcint] opened tunnel gcint:27017 <---> gc:local:8888
[gcint] [loadavg]
[gcint] 0.16 0.10 0.03 1/507 1273785 diskspace: 17% MemFree: 303160 kB
```
hieronder sshtunnel zie je de selectors WEL opgegeven via de cmdline (dus hij prompt niet)
## SSH tunnel / Mongodump
Hiermee ben ik een eind gekomen, en dat werkt 'sortof':
```
$ ./gc server=gc:int port_local=8888 port_remote=27017 sshtunnel loadavg
[gcint] [sshtunnel]
[gcint] closing (previous) sshtunnels..
[gcint] 17507
[gcint] 17547
[gcint] $ ssh groundcontrol@gcint -fN -L 8888:127.0.0.1:27017
[gcint] opened tunnel gcint:27017 <---> gc:local:8888
[gcint] [loadavg]
[gcint] 0.15 0.10 0.03 1/504 1273687 diskspace: 17% MemFree: 306092 kB
```
Het finetunen wil ik doen a.d.v. de docker-in-docker / next step:
## Next step: highlevel 'sync' commando prototypen dmv. bestaande sub commandos
Wellicht gewoon 'sensen' adv. current directory? (`$ pwd`) :
```
$ cd acc
$ ./gc sync
$ cd -
$ cd live
$ ./gc sync
```
een dergelijk highlevel multi-server 'story' kunnen we nu makkelijker prototypen:
```
sync(){
docker-compose down
_str_match $(pwd) "acc" && _gc from:gc:int to=gc:local/acc src_sync db_sync
_str_match $(pwd) "live" && _gc from=gc:local/acc to=gc:local/live src_sync db_sync
docker-compose up -d
}
```
> de `db_sync` verschilt per case, die we a.d.v. selectors `db_strategy=merge` en `collections=*,-state` e.g. kunnen finetunen.
Ook kunnen we evt. aliases maken voor de `collections`-selector (voor `db_sync`):
* collections=runtime --> `collections=messages,answers,*_states,*_logs`
* collections=runtime,users --> `collections=messages,answers,*_states,*_logs,users`
## call 04-11-2021
```
4:35 <LeonvanKammen> Joost Lubach
14:35 <LeonvanKammen> 2:34 PM
14:35 <LeonvanKammen> gc sync gc:local gc:acc
14:35 <LeonvanKammen> gc sync . gc:groundcontrol/acc
14:35 <LeonvanKammen> gc sync /Users/joost/Work/groundcontrol gc:groundcontrol/acc
14:35 <LeonvanKammen> Joost Lubach
14:35 <LeonvanKammen> 2:35 PM
14:35 <LeonvanKammen> gc:groundcontrol/live ==> gc:live
14:36 <LeonvanKammen> gc:htg/acc
14:37 <LeonvanKammen> gc:htg/live
14:52 <JoostLubach> `./gc dosomething arg1 arg2`
14:53 <JoostLubach> `./gc server=htg something arg1 arg2`
14:54 <JoostLubach> `./gc server=local something arg1 arg2 server=htg something arg3 arg4`
14:54 <JoostLubach> `./gc server=htg arg2 arg1 something`
14:55 <LeonvanKammen> FOO=1 BAR=1 ./doiets
14:55 <LeonvanKammen> ./gc foo=1 bar=2 doiets foo=2 bar=4 doiets
14:56 <JoostLubach> `./gc sync int_to_acc` (edited)
14:57 <JoostLubach> `./gc` executable
14:57 <> `sync` command
14:57 <> `int_to_acc` argument
14:57 <JoostLubach> `./gc VAR=1 sync int_to_acc`
14:58 <JoostLubach> `./gc`: executable
14:58 <JoostLubach> `VAR=1` env variable
14:58 <JoostLubach> `sync` command
14:58 <JoostLubach> `int_to_acc` argument
15:00 <JoostLubach> `./gc sync int_to_acc`
15:01 <JoostLubach> `./gc sync int_to_acc --stack htg --app mission` (edited)
15:02 <JoostLubach> `fastlane beta arg1 arg2 platform:android` (edited)
15:02 <JoostLubach> `./gc sync int_to_acc --stack htg --app mission`
15:03 <JoostLubach> `Usage: ./gc <command> [--stack=<stack>]` (edited)
15:03 <LeonvanKammen> `./gc sync int_to_acc --stack htg --stack foo --app mission`
15:03 <LeonvanKammen> `./gc ping`
15:03 <LeonvanKammen> `./gc ping --stack htg --stack foo`
15:04 <LeonvanKammen> `./gc sync gc:local gc:live/acc`
15:06 <LeonvanKammen> `./gc push int_to_acc`
15:06 <JoostLubach> `./gc deploy int_to_acc`
15:07 <LeonvanKammen> `./gc pull --stack htg`
15:07 <LeonvanKammen> ^-- boeiend
15:23 <JoostLubach> `./gc hotfix ENV` -> alle stacks, alle apps
15:23 <JoostLubach> `./gc hotfix acc`
15:23 <JoostLubach> `./gc hotfix live`
15:23 <JoostLubach> `./gc hotfix acc --stack=gc --app=mission`
15:24 <JoostLubach> `./gc hotfix live --stack=htg --stack=obm --app=web --app=mission`
15:25 <JoostLubach> `./gc sync int2acc`
15:25 <JoostLubach> `./gc sync acc2live`
15:25 <LeonvanKammen> 2 ipv _to_
```
Sign in with Wallet
Connect another wallet