# DTLN OpenVINO Support

https://community.rhasspy.org/t/dtln-noise-suppression-setup/2799/3

https://github.com/chungyehwangai/DTLN

Please refer to python package list

tensorflow 2.2
soundfile
wavinfo
keras2onnx
onnxruntime
numpy 1.18.5

git clone https://github.com/chungyehwangai/DTLN.git

cd DTLN
python convert_weights_to_ov.py -m pretrained_model\DTLN_norm_500h.h5 -t ov_model

cd ov_model_1
python "c:\Program Files (x86)\intel\openvino_2021.4.547\deployment_tools\model_optimizer"\mo.py --saved_model_dir . --disable_nhwc_to_nchw --model_name DTLN_norm_500h
cd ..

cd ov_model_2
python "c:\Program Files (x86)\intel\openvino_2021.4.547\deployment_tools\model_optimizer"\mo.py --saved_model_dir . --disable_nhwc_to_nchw --model_name DTLN_norm_500h
cd ..

python real_time_processing_ov.py -m1 ov_model_1\DTLN_norm_500h_4.xml -m2 ov_model_2\DTLN_norm_500h_4.xml -i audioset_realrec_airconditioner_2TE3LoA2OUQ.wav

Can't run on GNA @@

python real_time_processing_ov.py -m1 ov_model_1\DTLN_norm_500h_4.xml -m2 ov_model_2\DTLN_norm_500h_4.xml -i audioset_realrec_airconditioner_2TE3LoA2OUQ.wav -d GNA
Traceback (most recent call last):
  File "real_time_processing_ov.py", line 48, in <module>
    exec_net1 = ie.load_network(network=net1, device_name=args.device)
  File "ie_api.pyx", line 367, in openvino.inference_engine.ie_api.IECore.load_network
  File "ie_api.pyx", line 379, in openvino.inference_engine.ie_api.IECore.load_network
RuntimeError: [GNAPlugin] in function void __cdecl GNAPluginNS::GNAPlugin::LoadNetwork(class InferenceEngine::CNNNetwork &): The plugin does not support layer: StatefulPartitionedCall/model_1/lstm_5/StatefulPartitionedCall/while:Loop

MODEL 1

MODEL 2


To train this model, follow steps in

https://github.com/breizhn/DTLN

https://github.com/breizhn/DNS-Challenge

python noisyspeech_synthesizer_multiprocessing.py
...
Generating file #59999
Of the 63498 clean speech files analyzed, 0.2% had clipping, and 23.3% had low activity (below 60.0% active percentage)
Of the 65706 noise files analyzed, 17.1% had clipping, and 0.0% had low activity (below 0.0% active percentage)

split_dns_corpus.py

### Debug with example LSTM network


https://keras.io/api/layers/recurrent_layers/lstm/

https://keras.io/guides/working_with_rnns/

https://stackoverflow.com/questions/66207609/notimplementederror-cannot-convert-a-symbolic-tensor-lstm-2-strided-slice0-t/66207610

# defining default parameters
self.fs = 16000
self.batchsize = 32
self.len_samples = 15
self.activation = 'sigmoid'
self.numUnits = 128
self.numLayer = 2
self.blockLen = 512
self.block_shift = 128
self.dropout = 0.25
self.lr = 1e-3
self.max_epochs = 200
self.encoder_size = 256
self.eps = 1e-7

>>> inputs = tf.random.normal([32, 10, 8])
>>> lstm = tf.keras.layers.LSTM(4)
>>> output = lstm(inputs)
>>> print(output.shape)
(32, 4)
>>> lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True)
>>> whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)
>>> print(whole_seq_output.shape)
(32, 10, 4)
>>> print(final_memory_state.shape)
(32, 4)
>>> print(final_carry_state.shape)
(32, 4)

### Python Package List (OpenVINO 2021.4.547 Windows 10)

pip list
Package                Version
---------------------- ------------
absl-py                0.12.0
appdirs                1.4.4
astunparse             1.6.3
attrs                  17.4.0
audioread              2.1.9
cachetools             4.2.2
certifi                2021.5.30
cffi                   1.14.5
chardet                4.0.0
decorator              4.4.2
defusedxml             0.7.1
ear                    2.0.0
enum34                 1.1.10
fire                   0.4.0
gast                   0.3.3
google-auth            1.30.1
google-auth-oauthlib   0.4.4
google-pasta           0.2.0
grpcio                 1.38.0
h5py                   2.10.0
idna                   2.10
importlib-metadata     4.4.0
joblib                 1.0.1
Keras-Preprocessing    1.1.2
keras2onnx             1.7.0
librosa                0.8.1
llvmlite               0.36.0
lxml                   4.6.3
Markdown               3.3.4
multipledispatch       0.6.0
networkx               2.5.1
numba                  0.53.1
numpy                  1.18.5
oauthlib               3.1.1
onnx                   1.9.0
onnxconverter-common   1.8.1
onnxruntime            1.7.0
opencv-python          4.5.2.52
opt-einsum             3.3.0
packaging              20.9
pip                    20.1.1
pooch                  1.3.0
progress               1.5
protobuf               3.17.1
py-cpuinfo             8.0.0
pyasn1                 0.4.8
pyasn1-modules         0.2.8
pycparser              2.20
pyparsing              2.4.7
requests               2.25.1
requests-oauthlib      1.3.0
resampy                0.2.2
rsa                    4.7.2
ruamel.yaml            0.17.7
ruamel.yaml.clib       0.2.2
scikit-learn           0.24.2
scipy                  1.4.1
setuptools             47.1.0
six                    1.16.0
SoundFile              0.10.3.post1
tensorboard            2.2.2
tensorboard-plugin-wit 1.8.0
tensorflow             2.2.0
tensorflow-estimator   2.2.0
termcolor              1.1.0
test-generator         0.1.2
threadpoolctl          2.1.0
typing-extensions      3.10.0.0
urllib3                1.26.5
wavinfo                1.6.2
Werkzeug               2.0.1
wheel                  0.36.2
wrapt                  1.12.1
zipp                   3.4.1