Moonshine

tags: `sketchdance`

Moonshine is a strongly typed text-based language intended to be represented well and completely by blocks, to be implemented easily in JavaScript, but compilable to native code efficiently. It is one piece of a project that includes Tardigrade (the block representation), Bootleg (the research OS), Moss Piglet (the UI), and Waterbear (that ties these all together).

Waterbear - the official Moonshine IDE incorporating blocks
Bootleg - experimental OS
Tardigrade - the block toolkit
Moss Piglet - the IDE toolkit
Moonshine - language that maps cleanly to and from blocks
Alley - server component to break scripts out of the browser (for Arduino, shell scripts, Minecraft, etc.)
Xastle - social media layer
Moat - secure anonymous file transfer
SketchDance Suite - web site combining Waterbear with other tools

Moonshine

More syntax refactoring ideas

The Scratch web forum uses "scratchblocks" to display "blocks" in forum messages.

say [Hello World!] for (2) seconds

Will create a step block with 2 inputs, one for text and one for numbers.

You can simulate drop-down lists by appending v to an input:

replace item (42v) of [Numbersv] with [The answer!]

And putting a hexcode in square brackets will show a numeric input

set pen color to [#ff0000]

Predicates use angle brackets:

if <((2) + (2)) = [5]> then
  say [Freedom is slavery.]
else
  say [Peace is war.]
end

Reporters use (overload) parentheses

wait ([sin v] of (90)) secs

Variables also use parenthesis

change y by (gravity)

this technique has a lot of built-in knowledge about Scratch, so that inserting symbols has several different shortcuts, and it seems to know which names correspond to "C-blocks" (contexts and triggers in WB). The end keyword marks the end of containment in a C-block, but there is no marker for the beginning of containment besides the name of the block and knowing whether or not that name is a container.

Up until now it has only been using existing block names and the block type and colour have been inferred. For custom blocks it gets more interesting.

Scratch has 11 normal categories of block, along with "grey", "extension", "custom-arg", and "obsolete". To colour a block, you append :: category

set [var v] to (1)           // normal
set [var v] to (1) :: motion // as a motion block

You can use any of the scratch shapes (stack (step), hat (trigger), ring (N/A), reporter (value), boolean (value), cap (no slot for a following block)).

show :: hat

And C-blocks use curly braces to show containment

run script {
} without screen refresh :: control

You can show blocks inside other blocks

this is (a block :: sensing stack) :: custom-arg stack

So, what can I learn from this for Moonshine?

I don't need to tell the blocks what colour to be, they will be defined in a Moon unit and all blocks in a unit will be the same color. Outside the unit definition they will be implicit, like Scratch.
I do need to give a return type for each block
I may make a bigger distinction between defining blocks, i.e., a moon unit, and using blocks for writing scripts. I am more concerned with brevity when blocks are used than when they are defined, and the types of sockets will be implicit, so I don't need different bracket types for them.
In Moonshine I have Step, Value (including arguments, returns, and locals), Context, and Trigger blocks. There are also Params which come in various flavours for drop-downs, text/numeric inputs, checkboxes, colour pickers, date/time pickers, blocks, and lists. Params and returns may be late-typed unless I can figure out a way out of that. Maybe a block for asserting type: [Type Array(String[3])::Number] would mean: There is a type here which takes Arrays of Strings, the arrays are of length 3, and it returns a number. I don't know, maybe some things could be typed by the first argument you drag to them (like I did for earlier Waterbear) or maybe there is a dropdown to customize it first. Explicit is better, although the dropdown may get pretty complicated and require a pop-up for specifying the type in all its gory details (especially if we want to apply constraints like "An integer between 1 and 6").
Custom blocks can themselves have returns, i.e., a point can be used as a point, or you can grab just the x from it and use that.

What I don't want is for the syntax to be sub-optimal because I'm not good enough at writing parsers!

Refactoring Syntax

NOTE (2021-01-04): The syntax of Moonshine should be less influenced by Javascript and more by the structure of blocks. It should be as much as possible a literal text transcription of the blocks.

Right now I have enough of Moonshine working in the parser to generate steps (and blocks from steps) and make function calls. Having written a fair bit of Moonshine now to define blocks in Waterbear, I am not super happy with it so far. Below is an example.

sprite{

  // Update parser to accept list notation Image[]
  spriteFromImages:Sprite = (images:Image) => {
    Sprite.spriteFromImages(images)
  }

  copy:Sprite = (sprite:Sprite) => {
    sprite.copy()
  }

  accelerateBy:Sprite = (sprite:Sprite, vector:Vector) => {
    sprite.accelerateBy(vector)
  }

  setVelocityTo:Sprite = (sprite:Sprite, vector:Vector) => {
    sprite.setVelocityTo(vector)
  }

  getVelocity:Vector = (sprite:Sprite) => {
    sprite.getVelocity()
  }
}

As you can see, it is too limited (no way to specify an array of Sprites), too verbose (a lot of repetitive syntax just to call into a JavaScript library), and too crunchy (what purpose do the "=" and "=>" signs serve besides imitating JavaScript?). It also does not cover the requirements for other blocks.

Step: has a namespace, a name, a return type, (return value name), typed, named parameters, and a body
Action: (AKA Event or Trigger) Has no return type or value, but is otherwise syntactically identical to a step.
Track: (AKA Context or C-Block) A container for one or more lists of steps. I could break from Scratch and make Actions be Tracks with no following slot. Has a return type and named value, a name and namespace, typed, named parameters, and one or more lists of blocks with selectors for which list to choose.
Value: The simplest blocks, used as "variables", constants, return values from other blocks, and parameters. Can be dropped by themself in a list of blocks to give a name to them, or to return them if they are the last value of a step.
Maybe some specialized steps for making new blocks, arrays, structs and for blocks whose type is indeterminate until params are known.
Should callbacks be a special kind of block? What about blocks that return a value of block?

Proposed syntax (updated 2020-12-22):

namespace{
     defs{
       /* values used by blocks, but not themselves exposed, mainly lists for
          select arguments */
       name:Type <= value
     }
    // step (most common block, least extra syntax)
    name:returnType["optionalReturnTypeName"](param:Type[=optionalDefault],param:Type[=optionalDefault]){
      body
    }
    // trigger, async, so no return type, no parameters
    trigger name(){
      body
    }
    // return type of a context is the return type of its last step as an array
    // or can it vary more than that?
    // or should context have a return type at all?
    // for now, no return type, let's see how that works
    // context
    // was: name["optionReturnTypeName"](param:Type[=optionalDefault]){
    context name(param:Type, param:Type){
      locals{
        /* Just like value blocks */
        name:Type <= value
        name:Type <= value
      }
      body
    }
    // value
    name:Type <= value
}

I think this captures the direct needs of Moonshine. Some things will get bumped to the Waterbear level: Not allowing names to be shadowed, generating lists of types (all directly defined types, lists of all directly defined types, lists of lists, blocks of various signatures), mutating return types. Some of this may move back down into Moonshine, like having variables declared in the Setup or Preload blocks be available to all other blocks, but it's a Waterbear-level thing to automatically include URLs in the Preload block when referenced elsewhere in the script, unless we're using Scene blocks (then each Scene can have its own Preload).

I still need to address how repetitive & verbose the body is, but maybe I can actually write some WB scripts first.

There also may be a higher level of objects than Sprites. Something like the avatars in RenPy, or Storytelling Alice / Looking Glass with sprites that have built-in social interactions. Ken Perlin was doing something with this too. This is probably a "for later" thing.

Examples Needed (as of 2021-12-09)

There are over 200 examples for P5: https://p5js.org/examples/
And more examples embedded in the docs
Over 100 examples for Pixi: https://pixijs.io/examples/#/demos-basic/container.js

Next Steps (as of 2020-05-26)

Current development happening on Glitch: https://glitch.com/edit/#!/waterbear

Data model of AST for Moonshine 0.1

{
    namespace: {type: 'namespace', name, values: []},
    literalValues: [Integer, Float#, Boolean, String, Array, Dict, Set, Vector],
    function: {type: 'function', name, params: ['0 to 3 typed names'], returnType},
    functionCall: {type: 'functionCall', namespace, name, args},
    comment: {type: 'comment', value},
    array: {type: 'array', value: []},
    dict: {type: 'dict', value: {}},
    string: {type: 'string', value: ''},
    boolean: {type: 'boolean', value: true || false},
    integer: {type: 'integer', value: 0},
    float: {type: 'float', value: 0.0},
    keyedValue: {type: 'keyedValue', object, key},
    indexedValue: {type: 'indexedValue', object, index},
    typedName: {name, type},
}

Update 2020-05-25

Yesterday I finally gave in and wrote a parser for the language that will become Moonshine. There are things I want (types, guards) that aren't supported by a pure subset of JS, and neither Lisp nor Forth felt like the right syntax for me, especially since neither carry type information.

Keeping it simple (and fairly verbose) for now, will work on shortcuts later, maybe.

Some notes from my journal

Blocks/text interchangeable
Always saved
Optionally collaborative
Refactoring supported and suggested
Primitives to support auth and private data
Small group forming
Code of Conduct built in

Somewhere between a MUD and a code editor: Garden of Waterbears.

Minimum set of primitives to build Moonshine from?

Name-binding
Functions/Generators/Continuations
Decisions
Iteration
Non-locality
Basic operators
Closures
Immutablility

Missing features (2019-05-08)

Zooming
Define new blocks
Define new types (structs)
Typed lists
More scratch-like (multiple scripts)
GUI to tweak values (both Bret Victor style and using forms)
Debugger/stepping
Import libs, get new blocks
Duplicating some Scratch blocks

IronPython DLR has the following messages (Small-talk style)

Get | Set | Delete Member
Get | Set | Delete Index
Invoke, Invoke Member
Create Instance
Convert
Unary Operation
Binary Operation

Specific language for WB
Subset of ES6?
Lisp-like syntax?
Structured as arrays
Prefers immutable via Mori (or something like it)
One include to be able to run scripts standalone (in browser or Node)

I've been struggling with whether to define Moonshine as JSON, a subset of JavaScript, or it's own format that I would have to parse. Part of the problem may be that I'm conflating two very different things: the definition of blocks (needing to specify arguments, types, etc.) and the specification of scripts (which assemble the blocks, but can also create new blocks, so there is legitimate overlap).

Since a block needs a lot of context, it can get more verbose. I need to specify:

Type returned
Arguments and their types and optionally defaults
Name
Help strings
Actual code
Constraints (number must be a positive integer, strings must be at least characters, arrays of two items, etc.)

Whereas for most scripts what I need is

Actual Code (which should probably include some type information, but is mostly wiring up blocks togther. I need enough type info to disambiguate blocks

I have probably also been overthinking using immutable state to drive building the UI and how that will play with transient state (dragging, open/closed, scrolling, resizing). It is fine to keep transient state in the DOM, that's where it belongs. Some things (resizing workspace) can be saved to localStorage. It is also fine to have different immutable state roots for different things: the block menu is essentially a separate app from the script blocks, and that's OK.

Going to try putting this into practice RSN.

What this could look like for block definitions:

// Multimethod

zero or more predicates (simplest predicates are type checkers)
pure function to apply
return type

// Commutative: order doesn't matter, only works with two arguments for now
Method('add').commut(isNumber, isNumber, (a,b) => a+b, 'number');
// Non-commutative, order is important
Method('add').when('vector', 'number', (a,b) => new Vector(a.x+b, a.y+b));
Method('add').when('array', 'array', (a,b) => [a…, b…]);
Method('add').when('array', 'any', (a,b) => [a…, b]);

OK, the more I think about it the more I dislike Multimethods in practice, are least for Waterbear. It is easy enough to name add_vector_number(a,b) to look like "Add" in the block (in a way that can be localized), but for writing scripts, and for human disambiguation it is better to have them namespaced or just named significantly.

i.e.,

Vector.prototype.addNumber(x)

Vector.add_Vector_Number(v,x)

are both preferable to

add(v:Vector, x:Number):Vector

because when code uses them you can see what is going on, and parsed code is easier to disambiguate. Of the two, I think I like the class syntax vs. the namespace syntax because it is shorter, keeping a balance between conciseness and clarity. But it does rely on this and doesn't map to blocks well, so back to namespaced? Maybe

Vector.addNumber(v,x) where Vector is a namespace, not a class (or a class acting as a namespace), but like a Python class, all namespaced function expect the named type as first argument.

In general, if functions return different things, they should be different blocks.

Which opens up a can of worms around loopOver(). Do I keep it as-is and loop over integers, objects, and arrays, or do I create three loop blocks? I think for maximum clarity, three loop blocks is the way to go.

Forth?

Forth might be a better model than LISP

There are already Forth implementations on bare metal to get a language, an OS, libraries, and and editor
There are typed versions of Forth (Factor)
Jones Forth is not a minimal Forth, and it does rely on Linux system calls, but it is an example of building Forth up from assembly language

Tagged Objects

(historical, I'm not 100% sure what I meant by this)

Just like JavaScript String Templates can be tagged to be run through the named function, I think something like tagged JSON would be useful to instantiate as an instance of the named object. The main question is: should it be valid JSON by incorporating the tag as part of the structure, or should it be external like the functions in String Templates?

While I would really like to move the tag outside of JSON for Moonshine, I think keeping JSON parseable by other tools and validators is probably more valuable than that nicety.

Moss Piglet

Runtime
Translate language <-> blocks
Run for speed or debugging
Tools for building languages on

Tardigrade

IDE
UI Components
Blocks
Cut & Paste

Alley

Optional backend to connect beyond the browser

One approach

Start with Immutable.js and Typescript
Convert mutable JS methods to immutable, remove mutable from namespace
Add functional libraries as needed
Start using it, see what else can be stripped out

Current areas of research

Continuations, coroutines, generators
Asynch mechanisms

"in most Smalltalk environments, the execution stack is a first-class citizen, coroutines can be implemented without additional library or VM support."

AFAICT Scratch uses the Actor model for concurrency and limits nearly all actor interactions to message broadcasts.

How compatible are all of these mechanisms with immutable state? And how well will any of this work with WebWorkers?

All built-ins should have attributes:

Mutable or immutable
Pure function?
stateful
Does it touch i/o
Big-O
WebWorker safe

Important considerations

transducers
immutable
strings based on regex/snobol
types pattern matching
vectors
dates
better floats than IEEE (i.e., Interval Arithmetic like in Frink, see https://github.com/Jeff-Tian/JavaScriptIntervalArithmetic)
fs
network
graphics
text
shell
processes (erlang-style) or go-routines?
piranha parallelism
crash-only
storage (tuplespaces)
prevaylent persistence
audio (like sonic pi)
pervasive lazy streams: key presses, mouse, network, time, files, …
units
prevent or warn of variable shadowing
patterns, guard clauses, constraints
decorators
vm never needs to stop: can hot load (and reload) libraries, even main code
vm can auto-discover other vms running and talk to them
object can have pointers to other objects that are anywhere on the internet
supports component-based views rather than app-based views (OpenDoc)

Goals for moonshine

graphics
games
writing
music / synth
surf web
waterbear
robotics, physical computing
bret victor-type widgets

Bootleg OS tools

Toggle between REPL and editor in shell like Pico-8
Card-based UI
Mouse/pointer works in shell
Markdown as pervasive hypertext

Alternative approach

Just use Elm which already does most (all?) of that

Why not Elm

Weird syntax
Need a server to compile
No changes at runtime
But can load Elm libraries at runtime from JavaScript
Really want web-based Elm compiler
- There is a rough compile of the Haskell Elm compiler to JavaScript, it is about 11 MB
- Between this and ports I could conceivably use Elm for Waterbear
- But for Bootleg, Elm really isn't client-side friendly, and maybe not the greatest for JavaScript games, Glitch integration, etc.

So, for now let's keep it in JS (or TS).

First steps:

Load default libraries: Immutable.js, Popmotion.js (https://popmotion.io/)
[Does Glitch do TypeScript?]
Processes
Shell
Graphics

All system calls:

fork() Create a process
exit() Terminate the current process
wait() Wait for a child process to exit
kill(pid) Terminate process pid
getpid() Return the current process’s pid
sleep(n) Sleep for n clock ticks
exec(filename, *argv) Load a file and execute it
sbrk(n) Grow process’s memory by n bytes
open(filename, flags) Open a file; the flags indicate read/write
read(fd, buf, n) Read n bytes from an open file into buf
write(fd, buf, n) Write n bytes to an open file
close(fd) Release open file fd
dup(fd) Duplicate fd
pipe(p) Create a pipe and return fd’s in p
chdir(dirname) Change the current directory
mkdir(dirname) Create a new directory
mknod(name, major, minor) Create a device file
fstat(fd) Return info about an open file
link(f1, f2) Create another name (f2) for the file f1
unlink(filename) Remove a file

From "A Constructive Look at TempleOS"

HolyC is both the system language and the shell script
- Replace with Moonshine, but same idea
No built-in calculator, just use shell
- Good, make sure basic math works. This means infix notation.
Use shell as REPL to build programs
- Yes, good. This means code which has been compiled in the shell is available to move to a file as needed. Make it accessible.
Menu file in home directory, always accessible via Ctrl-M. Edit this file to create any kind of launcher
- Handy. What kind of menu? Top=line? Context-sensitive? May have to fire up TempleOS to see this.
Most files auto-save on exit
- Good, unless there is a really compelling reason not to. Save undo stack too? Have a special way to exit without saving?
System-wide autocomplete. Hit Ctrl-F1 to get a list of completable words including filenames, symbolnames, etc. All source code is indexed and you can jump to any function from anywhere, even in the shell
- Interesting use of global space.
Unified hypertext everywhere, including shell. From command line Uf('foo') will disassemble a function and each symbol will be hyperlinked to source
- For now not worried about disassembling or JIT-compiling, but linking symbols to source (AND TO USE) is what Dexter was intended to be, and still a good idea.
Type() displays files like cat, respecting hypertext and displaying graphics
In shell, Dir gives directory listing, click on any folder to go to it
DolDoc is the the ubiquitous hypertext system. Ctrl-T gives raw text source. Can contain text, images, 3D meshes, macros (system commands that run on click). Similar to Oberon. This replaces HTML, JSON, XML, shell scripts, sourcefiles, text files
A hypertext link can be to a grep-search so results like a file listing always be up to date
Can embed images and flowcharts in source code. Flow chart boxes can also be hyperlinked.
Ctrl-R can be used at any point to bring up resource editor that lets you draw things. Sprites you draw are embedded directly in ddocument and can be referred to by numbered tags
HolyC has no main(). Anything at top-level scope is executed as it runs through compiler
Every piece of code (except kernel/compiler) is JIT-compiled on demand
Tag a function with #help-index and it will automatically appear in the docs
#exe compiler directive to shell out to external commands and include their output in source code while compiling
Full metadata and reflection support. Any class can be enumerated to get names, etc., but can also have arbitrary metadata attached
last class keyword as default argument. Provides name of previous argument's type as a string
No ahead-of-time linker or object files
Symbol table accessible at runtime
No environment variables, just use regular variables
No formal build system, just compile a file
Hit F5 in editor to JIT-compile and run
Call Dbg() or use Alt-Ctrl-D to get a debugger which still runs as a full REPL
One call to create a window context to draw into
No hardware support. VGA-only. No soundcard besides PC speakers. No networking.
No memory protection
No file permissions
No threads, just processes

Moonshine

A language based on Javascript

Moonshine is a subset of Javascript ES6 plus libraries, Sweet.js macros, and a coding style based on functional programming.

From ES6

Arrow functions
Block scope (let, const, functions)
Classes
Default parameters
Destructuring
For-of loops (iterators)
Generators
Modules
Rest parameters
Spread operators
Template strings
New methods on String and Object
Enhanced object literals
Comprehensions
Map, set, weakmap, weakset
Proxies
Symbols
Subclassable built-ins
Promises
New methods for Math and Number
Binary and octal literals
Tail call optimization

From Sweet.js

macros
syntax for other new features

Other libraries as “built-ins”

P5.js or similar
Mori.js for immutable structures
document-register-element.js / DOM4.js
localForage for local persistence
hoodie or firebase for remote persistence
csp.js for go-like channels
GA game library? https://github.com/kittykatattack/ga

Still need:

A better wrapper for canvas/SVG (MathBox2? Pixi.js?)
A low-level abstraction of CSS
A wrapper or alternative to Regex. Candidates: re-build (https://github.com/MaxArt2501/re-build), apg-exp (https://www.sitepoint.com/alternative-to-regular-expressions/), OMeta.
Modules, library loading
UI library

2016-08-07 Thinking

Keep it minimal. What can we take away from JavaScript?

Arrays
Objects?
Optional syntax
Assignment
Statements? (only expressions)

Keep

Number
Strings
Arrow function expressions
Some or all of standard library

Time to put some code down:

Define the starting syntax
Pull in some libraries
Write some examples, starting with syntax validation

Syntax:

list -> value [, list]
number -> (this is actually fairly complex)
" -> *characters + "
' -> *characters + ' // do we need both?
([val [,val]) => expression
tuple -> (val [,'val'])
name + tuple -> function call

APL Summary:

Everything is in arrays
Data can be strings or characters
A single item is a scalar
A one-dimensional array is a vector
A two-dimensional array is a matrix
Higher dimensional arrays are just arrays
Each data item in an array can be a character, a number, or an array
Arrays which contain both numbers and characters are “mixed arrays”
Arrays which contain other arrays are “nested arrays”
Some versions of APL support complex numbers
Some versions of APL support collectionss of data and/or functions called Overlays
In object-oriented APL a data item can be either a class reference or an object reference
Data items can be given names, in which case they are variables
There are about fifty built-in operators and function, each is one character. Most can be called with either one or two arguments, providing different (but related) operations.
System variables and functions are an extension to the built-in set and always start with the quad “⎕” character to distinguish them
System commands are also extensions, mostly for working with the workspace (REPL) and start with ")". While usually typed directly, they can be used in scripts as well.
Functions can take 0, 1, or 2 arguments. Functions always take data as arguments. Functions act on whole arrays without need for explicity looping.
Operators are like functions, but their arguments (operands) can themselves be functions.
Assignment operator ←
Names for variables: APL uses upper-case and lower-case characters. Many APLs also allow the symbols Delta (∆), underlined Delta (⍙), the underline (_), and the high minus (¯)
⍴ (rho) is the shape operator
⍴ x => returns shape of x (number of rows and columns)
2 3 ⍴ 1 2 3 4 5 6 => returns matrix of 2 rows and 3 columns: [[1,2,3],[4,5,6]]
6 ⍴ 9 => single row containing 6 columns, all filled with 9: [9,9,9,9,9,9,]
⍴ can be used to create single-value vectors 22 is a scalar but 1 ⍴ 22 is a vector containing a single data value
Square brackets are used for indexing
Multi-dimensional arrays are indexed with multiple numbers x[1;3;6]
Multiple indexes from a single axis can be selected: x[1;1 2 3]
For all values in an axis, leave it blank: x[1;] (row) or x[;2] (column)

Attributes of code (objects) generally take the following forms (from Doug Lea):

instance variable
- direct
- indirect
  - repository to query
  - property list
derived

Categories of attributes:

Value representation
Aquaintance
History
Logical state / Role variables
Execution state
Cached representation

Moonshine

tags: sketchdance

Moonshine

More syntax refactoring ideas

Refactoring Syntax

Examples Needed (as of 2021-12-09)

Next Steps (as of 2020-05-26)

Data model of AST for Moonshine 0.1

Update 2020-05-25

For Social Programming (2019-05-06)

Minimum set of primitives to build Moonshine from?

Missing features (2019-05-08)

IronPython DLR has the following messages (Small-talk style)

Forth?

Tagged Objects

Moss Piglet

Tardigrade

Alley

One approach

Current areas of research

Important considerations

Goals for moonshine

Bootleg OS tools

Alternative approach

Why not Elm

First steps:

All system calls:

From "A Constructive Look at TempleOS"

Read more

Operating Systems

Sketchdance Suite

Bootleg OS

Waterbear

tags: `sketchdance`