# Martin Howse ###### tags: `PARTICIPANTS` ## martin development 27/11 Martin unable to come to taiwan due to health issues. He sent notes. https://hackmd.io/kFejznSlT3CVr_UxDID9Zw ## Martin sound notes ### Day One - Checking lists - we need small mixer and monitor speakers - Making test recordings with USB microphone: I suggest as many layers as possible to uncouple all, also placed on foam on concrete floor - We have problem with audacity not registering signals above 20KHz - check this with **baudline** and there we have a nice full spectrum and can also record files... - Working later on Python code which might allow bugs to compose the libretto - We need more soundproofing ![](https://i.imgur.com/P59ds3e.jpg) ### Day Two - Link for bug sounds: https://www.ars.usda.gov/southeast-area/gainesville-fl/center-for-medical-agricultural-and-veterinary-entomology/insect-behavior-and-biocontrol-research/people/richard-mankin/sound-library/#riceweevil We are interested in: Sitophilus oryzae ! - Piezo workings... trying also piezo radio transmitter - we will always have feedback issues so how to exploit these with the help of the bugs? ### Day Three TODO: test other radio transmitters, test divide down circuit See also for downsamplimg and filtering: https://tai-studio.org/index.php/2011/03/materials-for-tais/ some SC code to highpass and delay: ``` ( { var in; var signal; signal = AudioIn.ar(1); d=DelayL.ar( HPF.ar(signal, 4000.0, 40),2,2 ); //Out.ar(0,in); (d).dup }.play(s) ) ``` Some code to generate libretto from bug energy: ``` # for more recent NLTK which doesn't have bm=NgramModel(2,cry) # bigram model # this one is working! # fixed now for python3 # python3 bug_libretto.py # http://www.nltk.org/api/nltk.lm.html#module-nltk.lm # you may need to apply fix here to portaudio: https://stackoverflow.com/questions/59006083/how-to-install-portaudio-on-pi-properly # generates libretto from sound source energy of bugs from nltk import * from time import sleep import math import random import time #import serial import codecs import os import subprocess as sp from nltk.lm import MLE from nltk.util import bigrams from nltk.lm.preprocessing import padded_everygram_pipeline import unicodedata import struct import numpy as np import sounddevice as sd from scipy.signal import butter, lfilter def butter_bandpass(lowcut, highcut, fs, order=5): nyq = 0.5 * fs low = lowcut / nyq high = highcut / nyq b, a = butter(order, [low, high], btype='band') return b, a def butter_bandpass_filter(data, lowcut, highcut, fs, order=5): b, a = butter_bandpass(lowcut, highcut, fs, order=order) y = lfilter(b, a, data) return y def convert_accents(text): return unicodedata.normalize('NFKD', text).encode('ascii', 'ignore') #soundings INTERVAL = 1 CHANNELS = 1 RATE = 48000 CHUNK = int(RATE * INTERVAL) CHUNK=4000 sd.default.samplerate = RATE sd.default.device = 3 # this is the mic #letterings #f = codecs.open("/root/projects/archived/worms_txt/chants","r","utf-8") f = codecs.open("/root/notes_and_projectsNOW/bug/dongio","r","utf-8") # wherever you have the source text file to get letter/letter frequencies from - longer is better. here don giovanni #f = codecs.open("/root/projects/archived/worms_txt/short") cry = f.read() #cry=[y.lower() for y in cry] #cry=[convert_accents(word) for word in cry] uf=FreqDist(l for l in cry) up=DictionaryProbDist(uf,normalize=True) #bm=NgramModel(2,cry) # bigram model #lm.fit(cry, vocab) #print list(bigrams(text[0])) #print list(bigrams(cry)) # what we need from text is: a text that is a list of sentences, where each sentence is a list of strings. # eg. text = [['a', 'b', 'c'], ['a', 'c', 'd', 'c', 'e', 'f']] sentt=[] for sent in sent_tokenize(cry): letterss=[] #sentt.append(sent) for letters in sent: letterss.append(letters) sentt.append(letterss) #print sentt #cryyy = list(cryy) #print cryy train, vocab = padded_everygram_pipeline(2, sentt) lm = MLE(2) lm.fit(train, vocab) #print lm.vocab #for prob? #print lm.score("a",list("b")) uf=FreqDist(l for l in cry) up=DictionaryProbDist(uf,normalize=True) #print uf #bm=NgramModel(2,cry) # bigram model #print lm.generate(5, text_seed=['c'], random_seed=3)) def genereate(dct,model,letter,n): probb=[] result='' total='' line='' for x in range(n): for l in dct.samples(): prob=model.logscore(l,tuple(letter)) # print prob, letter, l probb.append((prob,l)) probb.sort() probb.reverse() # print probb # line = ser.read(1) # line = random.randint(0,32) # line=1 # line is our line now spectral # problem is to overlap our chunks otherwise we have to wait so long data = sd.rec(CHUNK, samplerate=RATE, channels=1) sd.wait() # y = np.frombuffer(data, dtype='b') y=data oneD_array_of_amps_of_fiveHz_component = butter_bandpass_filter(y, 500, 1000, 4000, 5) #calculate energy like this energy_of_fiveHz_comp = sum([x*2 for x in oneD_array_of_amps_of_fiveHz_component]) nummm = int(1000.0*energy_of_fiveHz_comp) if nummm<0: nummm=2; if nummm>=len(probb): nummm=len(probb)-1; letter=probb[random.randint(0,int(nummm))][1] result=result+letter probb=[] if letter=="\n": return result[0:-1] return result[0:-1] inpt=time.time() #ser = serial.Serial('/dev/ttyACM0', 9600, timeout=1) seedletter='e' fullon=[] count=0 #pr=lm.generate(2000) # this seems to work with our model but not our generate which lacks spaces for one thing #print ''.join(pr) for x in range(100): x=genereate(up,lm,seedletter,64) # max length of line print(x) ``` More supercollider with downsampling: ``` ( o = Server.default.options; o.sampleRate=96000; Server.default.reboot; ) ( { var in; var bits=16; // bits=16, downsamp=2| var down; var downsamp=4; var signal; signal = AudioIn.ar(1); in = signal.round(0.5 ** bits); down = Latch.ar( in, Impulse.ar(SampleRate.ir / downsamp.max(4)) ); signal=blend(in, down, (downsamp - 1).clip(0, 1)); d=DelayL.ar( HPF.ar(signal, 4000.0, 40),2,2 ); //Out.ar(0,in); (d).dup }.play(s) ) ``` And borrowed from Fredrik LFSR with input: ``` ( var w, a; w= Window("lfsr", Rect(100, 100, 520, 200)).front; Slider(w, Rect(10, 10, 500, 25)).action_({|view| a.set(\rate, view.value*2000)}).value= 400/2000; Slider(w, Rect(10, 40, 500, 25)).action_({|view| a.set(\length, view.value*32)}).value= 16/32; a= {|rate= 400, iseed= 2r1000, tap1= 1, tap2= 3, tap3= 5, length= 16| var l, b, trig, o, signal, in; var buf= LocalBuf(1); buf.set(iseed); trig= Impulse.ar(rate); l= Demand.ar(trig, 0, Dbufrd(buf)); //read //b= l.bitXor(l>>tap1).bitXor(l>>tap2).bitXor(l>>tap3)&1; //modify signal = AudioIn.ar(1); in = signal.round(0.5 ** 8); //b= l.bitXor(l>>tap1).bitXor(l>>tap2).bitXor(l>>tap3).bitXor(in)&1; //modify b= (in)&1; //modify l= (l>>1)|(b<<(length-1)); //lfsr Demand.ar(trig, 0, Dbufwr(l, buf)); //write o= PulseCount.ar(Impulse.ar(rate*length), trig); //bits l>>o&1!2; //output }.play; CmdPeriod.doOnce({w.close}); ``` ### Day Four Streaming in for performance - I use remote jitsii stream from ricebugs in remote/quieter location with supercollider (code is more or less above with addition of sliders for volume and high pass frequency). Using pulseaudio (pulseaudio -vvv) and qjackctl with extra configuration in qjackctl - and then work with the routing of the pulseaudio sink and source https://www.celesteh.com/blog/tag/supercollider/ ### Day Five Flying bugs! Setup of remote stream (icecast?) from isolated bug location with USB microphone - also spectral capture. mplayer -nocache http://example.org:8000/node1 -> qjackctl -> sc ## Approaches ### Acts/ideas - FM bugs - bugging the bugs, transmitter bugs - bugs write the libretto (similar technique to working with worms here: http://www.1010.co.uk/org/worms.html but instead of tracking electrical change we could use energy in different frequency bands) - what could we use as the language model (don giovanni?) - in progress - bugs manipulate artificial voices, parameters (see in supercollider: https://composerprogrammer.com/teaching/supercollider/sctutorial/12.2%20Singing%20Voice%20Synthesis.html) - scheming with the bugs: bugs interactions with code - livecoding by the bugs, bug entry into coded structures like LFSR (https://fredrikolofsson.com/f0blog/linear-feedback-shift-register/) arecord -D hw:0 - Sampling and downsampling: mic->highpass->downsample - supercollider ### Technical - USB ultrasonic microphone and raspberry pi4 (software based) - Ultrasonic MEMS microphones and downconvertor - audible signals - Ultrasonic MEMS microphones and amplifier sampled by pi4 and audio codec - Raspberry Pi Shield - HiFiBerry DAC+ ADC Pro - Radio transmission from vibrations from the bugs - Audio communication to the bugs (pi based with small amplifier and Taro speakers) Notes: vibration damping from below, protection of sensitive mics from moisture/humidity, dealing with ultrasonic signals. Platform: pd, sc or custom... https://github.com/redFrik/supercolliderStandaloneRPI2 ### Experiments - how to capture and visualise spectrum of bug activity? https://www.baudline.com/download.html spectogram view (for track) in audacity sox sound.wav -n spectrogram (generates png) python and matplotlib https://medium.com/quick-code/python-audio-spectrum-analyser-6a3c54ad950 freqscope for sc: https://doc.sccode.org/Classes/FreqScope.html - install software for audio on pi - supercollider and jackd are installed, needs puredata or other platform - capture and playback ultrasonic audio from bugs (pi and/or laptop), examine feedback and vibration - generate high frequency signals from GPIO on pi and interface with speakers ### USB microphone Ultramic UM192K (https://www.dodotronic.com/product/ultramic-um192k/?v=2a47ad90f2ae ![](https://i.imgur.com/DE7RQ1u.jpg) Notes - the end has a cap and when removed will need to be protected from moisture/humidity without impacting too much one sensitivity. Same for all microphone elements. ![](https://i.imgur.com/6JWrp1r.jpg) ### MEMS microphones and downconversion ![](https://i.imgur.com/VUHqWyB.jpg) ![](https://i.imgur.com/GIBODGv.jpg) ![](https://i.imgur.com/w5SGmxW.jpg) ![](https://i.imgur.com/nrwFha0.jpg) ![](https://i.imgur.com/nUQ9TLl.jpg) ### Bug FM transmitters around 100 MHz The design has changed a bit - it can be cut on copper foil using a vinyl cutter and easily assembled. Very sensitive to vibration. Tune receiver on FM to 98-105MHz. I was also thinking some rice could be sprayed with metal copper and this could interfere with the transmission. Needs to be experimented with. ![](https://i.imgur.com/WHV4rj3.jpg) ### PI4 and HiFiBerry DAC+ ADC Pro audio shield ![](https://i.imgur.com/idJUBRv.jpg) 1 Analogue input, phone jack 3.5mm 2 Analogue output RCA 3 Analogue output (p5) 4 Input configuration jumper (J1) 5 Alternative input connector (P6) https://www.hifiberry.com/docs/data-sheets/datasheet-dac-adc/ ### Shipping/ordering Shipping list: - USB ultrasonic microphone: https://www.dodotronic.com/product/ultramic-um192k/?v=2a47ad90f2ae - Raspberry PI4 + 16GB sdcard (with pi os installed and audio configured) + power supply (EU plug) + HiFiBerry DAC+ ADC Pro audio shield - 2x MEMs microphones: one with downconversion to audible range and 1 with amplifiers for sampling with pi4+ADC (there were 3 but one didn't work - they are very cheap) - 4x printed radio transmitters - 1x piezo radio transmitter - some electronic parts for making amplifiers and transmitters Shopping list: - Copper spray like: https://www.solianiemc.com/en/p/electrically-conductive-spray-paint-400-ml/ >> similar product in Taiwan >> * [GRAPHIT 33](https://www.ruten.com.tw/item/show?22026775925827) >> * Cooper Spray [紫銅噴劑](http://www.wei-neng.com.tw/product_info.asp?id=256) - this one is good! - 2x **5v** DC 200mA+ power supplies with **2.1mm** plugs like: https://www.aliexpress.com/item/4000077220290.html?spm=a2g0o.search0302.0.0.64057e24rBv8eR&algo_pvid=082696f3-397d-4217-8787-8f130acef647&algo_expid=082696f3-397d-4217-8787-8f130acef647-20&btsid=2100bdd516069103488434312e3560&ws_ab_test=searchweb0_0,searchweb201602_,searchweb201603_ >> 電子材料行 - audio cables (1.5meters plus): >> 電子材料行 - 2x cinch/RCA to mixer/jack (DAC out): https://www.avacab-online.com/CORDIAL-CFU-PC-Jack-Mono-Male-to-RCA-Male-Cable/en >> 電子材料行 - 3x stereo minijack (3.5mm) to mixer/jack - 6.5mm 2xmono (mic amp to mixer, radios to mixer), 1x mono minijack to mixer/jack (downconvertor to mixer): https://www.av-connection.com/?PGr=11730 - 1x stereo minijack (3.5mm) to stereo minijack (3.5mm) - mic amp to pi ADC: https://www.av-connection.com/?ML=310 >> 電子材料行 >> - MEMS microphone (eg. SPU0410LR5H) on breakout board like: https://www.sparkfun.com/products/9868 >> [SparkFun MEMS Microphone Breakout – INMP401 (ADMP401) 麥克風感測器模組](https://www.taiwansensor.com.tw/product/sparkfun-mems-microphone-breakout-inmp401-admp401-%E9%BA%A5%E5%85%8B%E9%A2%A8%E6%84%9F%E6%B8%AC%E5%99%A8%E6%A8%A1%E7%B5%84/) - basic electronic parts: 2xMCP6002 amplifier (DIP), breadboard, 10x 100k resistors, wire, jumpers >> 電子材料行 - Raspberry PI3 + 16GB microSD card + 5v 2.5A power supply with MICRO-USB connector for PI. - 2x FM/AM radio recievers like: ![](https://i.imgur.com/Ndlk0rI.jpg) >> 電子材料行 - 4x 9v batteries, 2x AAA batteries >> 電子材料行