NIME 2020 "Support System for Improvisational Ensemble Based on Long Short-Term Memory Using Smartphone Sensor"

• Slack: https://app.slack.com/client/T0174GB8UBA/C017PKWLUGH

Q1

• Jason Hockman 今日 20:44
  Could you discuss the LSTM model in more detail? Perhaps the model architecture and relevant parameterisation?
• Haruya Takase 28分前
  @Jason Hockman LSTM is suitable for estimating time series data such as music and text data because having a layer that retains past output. Actually LSTM gives better accuracy in attack timing estimation and vertical motion estimation than Bayesian network in this result. I hope that answers your question.
• Jason Hockman 6日前
  @Haruya Takase Many thanks for your reply. Are you using the cell independently or within a layer of a network (e.g., RNN)?
• Haruya Takase 6日前
  @Jason Hockman We are using simpleLSTM from keras. Since LSTM is an extension of RNN, the memory cells are contained in the network layer. The difference between RNN and LSTM is that RNN simply keeps data up to the present, whereas LSTM has a forget gate to select the past data to keep. RNN causes gradient loss by using all the past data for the current training, but LSTM avoids this problem thanks to the forget gate.

Q2

• Tom Mitchell 今日 20:44
  Hi @Haruya Takase thanks for the talk really interesting - modern smartphones often incorporate barometers I wonder if you have considered using that for estimating vertical position (altitude)?
• Haruya Takase 1時間前
  I hadn't considered using a barometric sensor.
  We'll use your message to help us with future research.
  Thank you！

Q3

• Julian Vogels (Soundbrenner) 今日 20:44
  What is the application of the Kinect in this performance system? Does it have the same function as the smartphone?
• Quinn Jarvis Holland (she, they) 1時間前
  could be wrong but looks like it's ugmenting the phone data with a height value?
• Haruya Takase 1時間前
  I use Kinect to collect teacher data for Vertical motion estimation.
  The Kinect records the height of the hand holding the phone.
  The vertical motion estimation LSTM model estimates the height of the hand holding the phone.
  The sum of the actual values between attack timings is used to determine vertical movement.

Q4

• Quinn Jarvis Holland (she, they) 今日 20:44
  Interesting graph of the two axis of rotation for detecting attack. Do you think that has applications for other gyroscopic/accelerometer device mapping?
• Haruya Takase 40分前
  @Quinn Jarvis Holland (she, they) it's possible to use other method. Could you teach me the names of the other methods for gyroscopic/accelerometer device mapping? （編集済み）
• Quinn Jarvis Holland (she, they) 28分前
  I don't have any names for methods really. Just have used mapping "pitch" and "yaw" or x and z rotation to pitch as well as filter cutoff on a synthesizer.
  https://github.com/pccadaptiveinstrumentsteam/PCC-Adaptive-Instruments-Project my team project here <–-I like the inclusion of the kinect- we also could not get a gould altitude reading without augmenting it with something like that
  
• Haruya Takase 6日前
  @Jason Hockman We are using simpleLSTM from keras. Since LSTM is an extension of RNN, the memory cells are contained in the network layer. The difference between RNN and LSTM is that RNN simply keeps data up to the present, whereas LSTM has a forget gate to select the past data to keep. RNN causes gradient loss by using all the past data for the current training, but LSTM avoids this problem thanks to the forget gate.