---
RFC: RFC0017
Author: Robert Holt
Status: Experimental
Area: Domain Specific Languages, Object Schemas
Comments Due:
---

# Domain-Specific Language Specifications

PowerShell provides rich metaprogramming features with a natural-language-oriented
interface, but currently lacks a true, simple mechanism for language extensions and
keyword addition.

Users have already created their own domain-specific languages (DSLs) in PowerShell
to suit their needs using a variety of language features, however a canonical DSL
definition mechanism could standardize this, in addition to providing more natural
operability within PowerShell, such as:

* Syntax highlighting
* Autocompletion
* Semantic checking
* Better readability
* Reproducible, schema-based formats
* Informative error messages
* Less XML (in the case of `.ps1xml` files)
* Specification of parse-time functionality

While not allowing arbitrary changes to the PowerShell language, a DSL mechanism
would seek to let users define their own language functionality, especially with
respect to types, with a well-defined and standardized syntax.

## Motivation

As a PowerShell user, I can create a domain-specific language
to specify a (print format | xml document | object-type
resource configuration | testing setup) schema
using a familiar PowerShell syntax, so that I can repeatably
instantiate that schema and leverage PowerShell's semantic
functionality for field checking and autocompletion.

### Examples of Motivating DSLs in PowerShell

#### Examples pre-existing in PowerShell

* [PowerShell `configuration` modules](https://blogs.msdn.microsoft.com/powershell/2013/11/05/understanding-configuration-keyword-in-desired-state-configuration/) — DSC configuration modules (Note: this already uses the `DynamicKeyword` mechanisms)
* [Pester](https://github.com/pester/pester) — testing framework
* [psake](https://github.com/JamesKovacs/psake) — build automation
* [simplex](https://github.com/beefarino/simplex) — PowerShell providers
* [ShowUI](https://showui.codeplex.com/) — WPF UI specification
* [PowerShell Data Files](http://ramblingcookiemonster.github.io/PowerShell-Configuration-Data/) — PowerShell-native data specification format

#### Motivating candidates for porting to a PowerShell DSL

* [Types.ps1xml](https://msdn.microsoft.com/en-us/library/dd878306(v=vs.85).aspx) — type addition to PowerShell
* [Format.ps1xml](https://msdn.microsoft.com/en-us/library/dd878339(v=vs.85).aspx) — format/presentation specification for PowerShell types and resources

The goal, then, would be to create a standard mechanism for PowerShell that all of these DSLs could be written and maintained in.

## Specifications

This RFC proposes a PowerShell attribute based mechanism for defining DSLs in PowerShell.

Existing DSL keywords are defined as PowerShell functions. All the keywords are separate functions with no real structure, other than documentation, defining how they used together.
We schematize the Keyword by marking the function with the `[DslKeyword]` attribute. Within that attribute, properties of the DSL can be specified.

**Definition**:
Inner Keyword is a DSL keyword that should be defined within the script block of a DSL keyword.

A user defines their DSL by marking their keyword function with the `[Keyword()]` attribute. Within the function, functionality is defined in normal PowerShell code. Inner keywords are defined in nested functions inside the main keyword function. Users would be able to specify which nested functions are inner keyword functions.

Take this example DSL structure from Pester.

Describe "Add-Numbers" {
    It "adds positive numbers" {
        $sum = Add-Numbers 2 3
        $sum | Should -Be 5
    }
}

To prevent multiple implementations of keywords nested into one block, we define the keyword functionality outside the nested block.

function Describe {
    [DslKeyword(InnerKeywords = @('It'))]
    param(
        [Parameter(Mandatory = $true, Position = 0)]
        [string] $Name,
        
        [DslBody()]
        [scriptblock] $Body,
    )
    
    function It {
        [DslKeyword()]
        param(
            [Parameter(Mandatory = $true, Position = 0)]
            [string] $Name,
        
            [DslBody()]
            [scriptblock] $Body,
        )
        _It();
    }   
    _Describe();     
}

function _It()
{
    # It keyword implementation here
}

function _Describe()
{
    # Describe keyword implementation here
}

Users could also attach their own error handling function within their keyword, should the a run time error occurs when the structure of the Keyword is malformed.

function Describe {
    [DslKeyword(ErrorHandler = @('catchStructureError'))]
    param(
        [Parameter(Mandatory = $true, Position = 0)]
        [string] $Name,
        
        [DslBody()]
        [scriptblock] $Body,
    )
    function catchStructureError() {
        # error handling code defined here
    }
}

### Questions

My understanding of how this is going to work. Please correct me if I'm wrong.

DslKeyword is a PowerShell class that inherites Attribute, which the PowerShell runtime instantiates and invokes at run-time.

Assuming we find a way to see what is defined in the scriptblock, if a required InnerKeyword is not defined inside the scriptblock we throw a runtime error telling the user to define the inner keyword.

**Unknowns**
How can I be able to see the contents inside a scriptblock? ie. I want to check if the InnerKeywords are defined inside the scriptblock.
Thoughts: Would this be possible without hacking the parser? If not, with the `DslBody` attribute, could this be possible?

### End of Ivan's changes
## Alternate Proposals and Considerations

### Argument types for `Keyword` methods

Currently the `Keyword` base class specifies its arguments with types internal
to the PowerShell parser, but it may be preferable to not expose these internal types
as a user interface directly. However, since we wish to provide a customizable DSL,
most of the internals of a `DynamicKeyword` and its AST will need to be exposed in some
way, and `DynamicKeywordStatementAst` is exactly the type conceived to encapsulate
that information.

Conceivably an interface could be provided along the lines of:

PostParse(object[] positionalParameters, Dictionary<string, object> namedParameters)
{
    ...
}

However, this would limit the extent to which a user could customize their keyword
instantiation.

### Provided implementations for Keyword functionality

Further to the above point, keyword instantiation could be made simpler by providing
default virtual methods that simplify the keyword declaration process (for example
by parsing parameters). Users with more sophisticated needs could then override
these methods to suit their needs. This would provide a useful way to empower most
users to make simple DSLs while not precluding more enthusiastic language customizations.
Such methods may be out of scope for the current RFC.

### Bootstrapping the C# specification language for a PowerShell version

To extend the suggestions for ease of use above, the C# keyword definition language
could, in principle, be used to define a keyword definition DSL within PowerShell itself.
Such an implementation is likely out of scope for this RFC however.

### DSC modules and CIM configurations

Small DSL specification language already exists in PowerShell for DSC module
files (`.schema.psm1`) and CIM configurations. These may be of consideration
both as a guide for a more general implementation, and in terms of conflicts
and code duplication — although there are certain specific functionalities
that are tied to these language features.

### Attributes vs interface implementation for C# Specifications

In the current draft above, specification of a DSL keyword involves the redundancy
of both declaring the `[Keyword]` attribute and extending the `Keyword`
class. This is also the scheme used by `Cmdlet`, however for this reduced case it
may not be necessary.

### Runtime Behavior Specification

Currently there is no suggestion as to the design or implementation of specifying
the runtime behavior of a keyword -- what the keyword actually does when executed.
Runtime behavior can presently be added using PowerShell itself, but specification
in C# may be a better subject of a different RFC.

### Existing Implementations for Dynamic Keywords in PowerShell

The PowerShell parser and AST already support the concept of a
`DynamicKeyword`, which this proposal is built around. The following table
attempts to draw an equivalence between desired DSL features, current dynamic
keyword support, and proposed C# syntax:

| DSL Function | `DynamicKeyword` Property | C# Syntax |
| :--------------------- | :---------------------------------- | :--------------- |
| Keywords | `Name` | `Name = ...` attr |
| Nested keywords | Keyword stack | Inner class |
| Keyword use constraints | ? (Semantics check?) | `Use = ...` attr |
| Body syntax | `BodyMode` | `Body = ...` attr |
| Arguments | `DynamicKeywordProperty` | `[Property]` |
| Parameters | `DynamicKeywordParameter` | `[Parameter]` |
| Custom types (enums) | `DynamicKeywordProperty.Values` | `enum` |
| Parse time action | `DynamicKeyword.(Pre|Post)Parse` | C# code/interface |
| Semantics check | `DynamicKeyword.SemanticCheck` | C# code/interface |
| Runtime action | None | Out of scope |

-----
PowerShell Committee Decision

Voting Results

Jason Shirk: Accept

Joey Aiello: Accept

Bruce Payette: Accept

Steve Lee: Accept

Hemant Mahawar: Accept

Majority Decision

Commmittee agrees that this RFC is sufficient to move to experimental stage and begin prototype code for further feedback.

Minority Decision

N/A