# Choria External Agents Specification

We would like to enable people to write agents in any language, this is a draft specification to show how we'll implement that feature.

## Structure

External agents will live in the normal mcollective agent directories to ease deployment using the same tooling and have a format like:

-rwxr-xr-x 1 root root 315 Sep 12 11:18 helloworld
-rw-r--r-- 1 root root 819 Sep 12 10:40 helloworld.ddl
-rw-r--r-- 1 root root 915 Sep 12 10:40 helloworld.json

Here we have the `helloworld` agent with it's ruby DDL and it's Golang DDL both of which would be required for now but I suspect we'll make Ruby read the JSON ones as part of this effort.

## DDL

The DDL files are like any other however the JSON DDL need to have specific metadata to indicate to Choria that this is an external agent:

"metadata": {
    "name": "one",
    "description": "Manage Operating System Packages",
    "author": "R.I.Pienaar <[e-mail]>",
    "license": "Apache-2.0",
    "version": "5.0.0",
    "url": "https://github.com/choria-plugins/package-agent",
    "timeout": 180,
    "provider": "external"
}

Note the new `provider` property.

## Communications

The Choria Server will invoke the `helloworld` executable with the following:

* 3 arguments first the request file, reply file and the protocol, `helloworld /tmp/xxxx.request /tmp/xxxx.reply choria:mcorpc:external_request:1`
* Environment variable `CHORIA_EXTERNAL_REQUEST=/tmp/xxxx.request`
* Environment variable `CHORIA_EXTERNAL_REPLY=/tmp/xxxx.reply`
* Environment variable `CHORIA_EXTERNAL_PROTOCOL` that is either `choria:mcorpc:external_request:1` or `choria:mcorpc:external_activation_check:1`

Any lines written to STDOUT are logged at INFO level and lines written to STDERR are logged at ERROR level

The working directory will be the OS temp directory.

### Requests

The request will be passed via the `CHORIA_EXTERNAL_REQUEST` file and might look like this:

{
 "protocol": "choria:mcorpc:external_request:1",
 "agent": "helloworld",
 "action": "ping",
 "requestid": "034c527089f746248822ada8a145f499"
 "senderid": "dev1.devco.net",
 "callerid": "choria=rip.mcollective",
 "collective": "mcollective",
 "ttl": 60,
 "msgtime": 1568281519,
 "body": {
   "agent": "helloworld",
   "action": "ping",
   "data": {
     "msg": "hello"
   },
   "caller": "choria=rip.mcollective"
 }
}

This is `choria req helloworld ping msg=hello`, the `protocol` field here indicate it is a request.

### Reply

Replies are written to the `CHORIA_EXTERNAL_REPLY` file and should look like this:

{
  "statuscode": 0,
  "statusmsg": "OK",
  "data": {
    "result": "hello"
  }
}

### Activation Checks

These agents will be called at start time asking them if they should start or not, the request looks like this:

{
  "protocol": "choria:mcorpc:external_activation_check:1",
  "agent": "helloworld"
}

Replies should look like this:

{
  "activate": true
}

## Status

The POC at https://github.com/choria-io/mcorpc-agent-provider/pull/98 implements these features:

- [x] Finds agents in normal ruby directories
- [x] Activation Checks
- [x] Request and reply JSON
- [x] Standard auditing
- [x] Incoming requests should be validated by the DDL, defaults should be supplied
- [x] Logging via STDOUT and STDERR lines
- [ ] Authorization - go choria has no authorization yet
- [ ] Choria discovery which is based on PuppetDB will not find these agents

## Working external agent

### Behaviour

$ choria req helloworld ping msg=hello
Discovering nodes .... 1

1 / 1    0s [====================================================================] 100%

dev1.devco.net
   Result: hello


Finished processing 1 / 1 hosts in 480.772928ms

### /etc/choria/external/helloworld

This is a bit of a raw hack job, one might anticipate small helper libraries for ruby, go, python etc would exist to make this easier, but this shows the basics. Ben Roberts is already making such a small wrapper for python based on this spec.

#!/opt/puppetlabs/puppet/bin/ruby

require "json"

def empty_reply
  {
    "statuscode" => 0,
    "statusmsg" => "OK",
    "data" => { }
  }
end

# no specific error handling, non zero exit implies its not activating
def handle_activation(req)
  File.open(ENV["CHORIA_EXTERNAL_REPLY"], "w") {|f| f.print JSON.dump("activate" => true)}
end

def request_error(msg)
  rep = empty_reply
  rep["statuscode"] = 1
  rep["statusmsg"] = msg

  File.open(ENV["CHORIA_EXTERNAL_REPLY"], "w") {|f| f.print JSON.dump(rep)}
end

# here we error handle in the standard rpc way by setting statuscode and msg
def handle_request(req)
  rep = empty_reply
  rep["data"]["result"] = req["body"]["data"]["msg"]
  File.open(ENV["CHORIA_EXTERNAL_REPLY"], "w") {|f| f.print JSON.dump(rep)}
rescue Exception
  request_error("unknown error: %s: %s" % [$!.class, $!.to_s])
end

def dispatch
  req = JSON.parse(File.read(ENV["CHORIA_EXTERNAL_REQUEST"]))

  if req["protocol"] == "choria:mcorpc:external_activation_check:1"
    handle_activation(req)
  elsif req["protocol"] == "choria:mcorpc:external_request:1"
    handle_request(req)
  else
    raise("unknown protocol: %s" % req["protocol"])
  end
end

dispatch

### /etc/choria/external/helloworld.ddl

metadata    :name        => "helloworld",
            :description => "Hello World Agent",
            :author      => "R.I.Pienaar <[e-mail]>",
            :license     => "Apache-2.0",
            :version     => "0.0.1",
            :url         => "https://choria.io",
            :timeout     => 20

action "ping", :description => "Replies back to a request" do
    display :always

    input :msg,
          :prompt      => "Message",
          :description => "Message to sent",
          :type        => :string,
          :validation  => '^.+$',
          :optional    => false,
          :maxlength   => 150


    output :result,
           :description => "The result from the Puppet resource",
           :display_as  => "Result",
           :default     => "",
           :type        => :string

end

### /etc/choria/external/helloworld.json

{
  "$schema": "https://choria.io/schemas/mcorpc/ddl/v1/agent.json",
  "metadata": {
    "name": "helloworld",
    "description": "Hello World Agent",
    "author": "R.I.Pienaar <[e-mail]>",
    "license": "Apache-2.0",
    "version": "0.0.1",
    "url": "https://choria.io",
    "timeout": 20
  },
  "actions": [
    {
      "action": "ping",
      "input": {
        "msg": {
          "prompt": "Message",
          "description": "Message to sent",
          "type": "string",
          "default": null,
          "optional": false,
          "validation": "^.+$",
          "maxlength": 150
        }
      },
      "output": {
        "result": {
          "description": "The result from the Puppet resource",
          "display_as": "Result",
          "default": "",
          "type": "string"
        }
      },
      "display": "always",
      "description": "Replies back to a request"
    }
  ]
}