tags: FabAcademy FLU Future Learning Unit Fab Lab Barcelona MDEF

Livecoding as Human Interface

Citlali Hernández aka Turbulente

Xavi Domínguez aka er Lunni

Main Goals for today

1. Understand basic concepts to prototype an interface
2. Navigate the immensity of possibilities, tools, technologies, protocols and languages to design and implement interfaces.
3. To learn about the creative process of a transdisciplinary artist.
4. Discovering, understanding and practicing livecoding.
5. Tinkering Human Interfaces

Interface

Designing an Interface Application

Key questions

• Your interface application in 1 sentence
  Ex: A user interface that shows the value of a sensor in a website
• What is my budget ?
• How much time do I have?
• Is it a proof of concept ? Is it a prototype ? Is it the final application ?
• What technical knowledge do I already have? What other profiles do I need on my team?

Key Ingredients

Basic Concepts

Languages

The Hello World Collection

Protocols

A protocol is a set of rules that governs how data is transmitted and received over a network.
Internet protocols are the rules and standards that enable communication between devices over the internet.

Websocket

WebSocket is a communication protocol that enables two-way communication between client and server over a single, long-lived connection.
It allows for real-time data transfer between the two endpoints without the need for frequent HTTP requests.
WebSocket is commonly used in applications that require real-time updates, such as online gaming, chat applications, and stock market tickers.

OSC - Open Sound Control

Open Sound Control (OSC) is a protocol for networking sound synthesizers, computers, and other multimedia devices for purposes such as musical performance or show control

Tools, technologies, protocols and languages to design and implement interfaces

Miro

Projects

Coding Robots - Families - CosmoCaixa

Documentation
Repository

Ironskulls ( Azul Petroleo)

https://ironskulls.es/

Repository

Smart Bike Project

Reference: http://fabacademy.xavidominguez.com/final.html
Language:Arduino, App Inventor,Javascript, html5, php
Data Interfaces: MySQL, TinyDB
Networking Communication: Bluetooth JBtek-HC-05 Wireless

K3 - Parlant amb la Júlia

Try

Repository

Examples

SC + Processing ( Serial )

Repository

Processing + Arduino ( Serial Communication )

Input: Arduino with Mois Sensor
Output: Processing + Arduino LED
Communication: Serial Wired

Processing Code

import processing.serial.*;

Serial myPort;  // The serial port

PFont f;                           // STEP 1 Declare PFont variable

float r = 30;
float turn = 0;
float goldenR = (1 - sqrt(5))/2;
float h = 100;
int value = 1024;
boolean noWater = false;
String plantState = "";
void setup() {
  //fullScreen();
  size(700, 1000);
  background(0);
   
   f = createFont("Arial",16,true); // STEP 2 Create Font
   
   //printArray(Serial.list());
   
  // Open the port you are using at the rate you want:
  myPort = new Serial(this, Serial.list()[0], 9600);
}

void draw() {
  
  // Read String from serial
  if (myPort.available() > 0) {
    String inBuffer = myPort.readStringUntil(13);   
    if (inBuffer != null) {
     
      inBuffer = inBuffer.replace('\n', ' ').trim();
      value = int(inBuffer);
    }
      fill(0);
     rect(80, 80, 500, 150);
     
    textFont(f,16);                  // STEP 3 Specify font to be used
    fill(0,100,0);                         // STEP 4 Specify font color 
    
   if(value>900){
     plantState = "Plant NEEDS WATERING";
     noWater=true;
   }else{
     noWater = false;
     if((600<value)&&(value<900)){
      plantState = "Plant is DRY";
     }else{
      plantState = "Plant is HUMID";
    
    }
   }
    textAlign(CENTER);
    text(plantState,180,100);   // STEP 5 Display Text
  }
  
  turn += goldenR;
  float alphaBlatt = random(100);
  float alphaRand = random(100, 150);

  translate(width/2, height/2-h);

  noFill();
   if(noWater){
     stroke(120,120, 120, 120);
   }else{
     stroke(0, random(255), random(150), random(100));
   }
  
  bezier(0, 0+(height/2)+h, 0, random(0, 300), 0, random(0, 300), random(-300, 300), random(0, 350));

  stroke(0, alphaRand);
   rotate(turn);
  if(noWater){
    fill(120, 120, 120, 0);
  }else{
    
    fill(random(200, 255), 0, random(150, 255), alphaBlatt);
  
  }
 
  ellipse(0, 0, 50, random(100, 300));
  fill(0);
  ellipse(0, 0, 2*r, 2*r);

}

void keyPressed() {
  if (key == 's') {
    myPort.clear();
    myPort.write("1");
  }
}

Arduino Code

int val = 0;
void setup()

{

  Serial.begin(9600);

  pinMode(A0, INPUT);
  pinMode(9, OUTPUT); 

}

void loop()

{

  val = Serial.read();
  Serial.println(val);
  Serial.flush();
  
  int mois = analogRead(A0);
  Serial.println(mois);

  if(val>=1){
    digitalWrite(9, HIGH);  
  }else{
    digitalWrite(9, LOW);  
  }

  delay(2000);

}

A-Frame + NodemCU + WebSockets

Repository

APP Inventor + Bluetooth HC08 + Sensor

APP Inventor

Arduino Code

#include <SoftwareSerial.h>
SoftwareSerial BT1(4,2);      // RX, TX recorder que se cruzan

void setup()
   { Serial.begin(9600);
     Serial.println("Enter AT commands:");
     BT1.begin(9600);
     pinMode(A0, INPUT);
    pinMode(9, OUTPUT); 
    
   }

void loop()
   {  int mois = analogRead(A0);
    if (BT1.available())
          Serial.write(BT1.read());
          
           
      if (Serial.available())
         { char c = Serial.read() ;
           Serial.print(c);
           BT1.write(c);
           
         }
         BT1.print(mois);
         delay(1000);
   }

Grasshopper firefly

Acces to the repo with the files

• Firefly experiments link
• Firefly plugin for grasshopperlink

Simple serial writte

Using Grasshopper to Control a Pan / Tilt Servo from Andy Payne on Vimeo.

Simple serial read

Image processing

Kinect processing

Grasshopper 3d + Firefly hand tracking from nachetz on Vimeo.

Generative Design

Reference: http://www.generative-gestaltung.de/2/
Languange: p5.js

IoT Devices with IFTTT

Reference: http://iot.futurelearningunit.com
Language: Arduino
Device Interfaces: IFTTT
Data Interfaces: Google Sheets
Networking Communication:WiFI ESP8266

Python

Example file: here
Language: Python, web application
Device Interfaces: Serial
Data Interfaces: Pandas, CSV

Node-Red

User Interfaces: Node-Red
Device Interaces: MQTT

The super freak option

User interfaces: blessed-contrib
Language: ascii/ansi art and javascript

Livecoding

• TopLap

• Hydra

• Hydra API

• Hydra Book

• Hydra Docs

• Sonic Pi