owned this note
owned this note
Published
Linked with GitHub
# Findings on SIP communication on the BTicino C300X
## Goal of the document
The goal of this document is to describe the process of the communication between components using SIP calls on the BTicino (C100X and) C300X.
Furthermore we define the steps that we need to take to gradually build out a system that allows us to communicate with other clients (HomeBridge, HASS, OpenHAB)
## Disclaimer
I am not a hacker and have no experience in it. I love finding out how things are set up.
I don't know much, but I do know much about nothing.
If you break your system due to change you make, don't blame me.
## Contributors
* Thanks to R who was kind enough to join forces and work on this
* Thanks to Neo who was also candidate to work on this
## Terminology
This explains some terms that are used in either this document or log files.
* SIP: Session Initiation Protocol
* SIPs: SIP Secure = SIP extended with TLS
* flexisip: The SIP server
* awsm: The answering machine
* ops: openserver used for (?)
* intercom: We use this in this document to reference the C100X/C300X video unit.
## Steps
I am trying to gradually build a path to a system where we can call our intercom to peak and listen to the video and camera,
as well as a system where we can receive an incoming call from the unit (more or less how our Door Entry app works).
How we are gradually building this up, is also how the system is going to be integrated in a home automation platform.
The idea is to install transcoding of audio/video on "our" system and not hack it into the intercom.
So the SIP handling is installed additionally on "our" system.
This makes a loosely coupled system where everything audio/video related is handled by SIP communication.
The focus is on video/audio and call handling. Opening of doors/locks is *NOT* our scope (this is already partially achieved and being handled).
Step goals:
1. Make a call to the intercom: using a SIP client we attempt to achieve a call to the intercom
1. Starting with SIP (unsecured)
2. Enhancing this further with SIPs
2. Receive a call from the intercom: Receive a call on the SIP client when we "call home" or press the bell button
1. Starting with SIP
2. Enhancing this further with SIPs
3. Install a SIP server on a seperate host (e.g. where you run your Home automation platform), make the intercom talk to it
Using either something like a proxy flexisip service or installing a SIP client which will be used to connect it into our home automation platform.
4. Make a SIP call to the intercom from your home automation platform
Using "our" server either streams the video and audio to "our" channels (using ffmpeg, vlc, ...)
5. Using this SIP service integrate it into a platform (our SIP server or client triggers something in our automation platform)
It sends you a notification or just opens the door automatically for example.
## Current research for 1i
We assume you have succesfully flashed the adapted firmware from: https://github.com/fquinto/bticinoClasse300x to continue with this and have access via SSH.
I have decided to use Linphone (for desktop) to use as a client, you can install it on your desktop.
Fetch it at https://new.linphone.org/technical-corner/linphone?qt-technical_corner=2#qt-technical_corner
The first attempt to communicate on SIP level was to get access to the network ports (5060 SIP, 5061 SIPs).
You need to make `flexisip` listen on an outward facing interface (wlan0):
Mount read/write
```
$ mount -oremount,rw /
```
open /etc/init.d/flexisipsh and change:
```
case "$1" in
start)
start-stop-daemon --start --quiet --exec $DAEMON -- $DAEMON_ARGS --transports "sips:$2:5061;maddr=$2;require-peer-certificate=1 sip:127.0.0.1;maddr=127.0.0.1 sip:192.168.0.XX;maddr=192.168.0.XX"
;;
```
If you use a fixed IP for you unit (e.g DHCP reservations), you can put the fixed IP.
You can also let the script handle it (uses your current IP if dynamic):
```
start-stop-daemon --start --quiet --exec $DAEMON -- $DAEMON_ARGS --transports "sips:$2:5061;maddr=$2;require-peer-certificate=1 sip:127.0.0.1;maddr=127.0.0.1 sip:$2;maddr=$2"
```
TODO: We might need to change the main.py script from https://github.com/fquinto/bticinoClasse300x to ask to automatically expose daemon ports to the LAN
Next, We need to change the firewall. The default value is to `DROP` when it isn't matched in the firewall rules.
For simplicity we just `ACCEPT` on the three chains.
NOTE: Don't have it assigned a public IP ... or you will be hacked.
```
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
```
TODO: We might need to change the main.py script from https://github.com/fquinto/bticinoClasse300x to ask to automatically drop the firewall rules
YOU NEED TO DO THIS EVERYTIME YOU REBOOT YOUR DEVICE (to be fixed later to automatically flush them (?))
Next we want more logging!
In /home/bticino/cfg/flexisip.conf change 3 lines (make sure you create a backup of this file somewhere if you are scared).
Set `log-level` and `syslog-level` to `debug`
In `trusted-hosts` add the IP address of your Desktop client (where you installed Linphone client).
This makes sure we don't need to bother with the initial authentication of username/password (problem for later?).
```
[global]
...
log-level=debug
syslog-level=debug
[module::Authentication]
enabled=true
auth-domains=c300x.bs.iotleg.com
db-implementation=file
datasource=/etc/flexisip/users/users.db.txt
trusted-hosts=127.0.0.1 192.168.0.XX
hashed-passwords=true
reject-wrong-client-certificates=true
```
Next, reboot by typing `reboot`
Verify it is listening and the transport listens on your IP:
```
~# ps aux|grep flexis
bticino 741 0.0 0.3 9732 1988 ? SNs Oct28 0:00 /usr/bin/flexisip --daemon --syslog --pidfile /var/run/flexisip.pid --p12-passphrase-file /var/tmp/bt_answering_machine.fifo --transports sips:192.168.0.XX:5061;maddr=192.168.0.XX;require-peer-certificate=1 sip:127.0.0.1;maddr=127.0.0.1 sip:192.168.0.XX;maddr=192.168.0.XX
bticino 742 0.1 1.6 45684 8408 ? SNl Oct28 1:44 /usr/bin/flexisip --daemon --syslog --pidfile /var/run/flexisip.pid --p12-passphrase-file /var/tmp/bt_answering_machine.fifo --transports sips:192.168.0.XX:5061;maddr=192.168.0.XX;require-peer-certificate=1 sip:127.0.0.1;maddr=127.0.0.1 sip:192.168.0.XX;maddr=192.168.0.XX
```
Next we need to use the account name that our Door Entry app uses:
```
$ cat /etc/flexisip/users/users.db.txt
username-hotmail.com-93C0CF187E366B254@XXXXXXX.bs.iotleg.com md5:00411460f7c92d2124a67ea0f4cb5f85;
....
```
Check `Syncing of users` when this file happens to be empty.
You will find several entries that start with the email address you used in your Door Entry app, I used that one to test.
Interesting to see is that the MD5 does not match the MD5 of my password in the Door Entry app (I just did a quick check, so they might be mangled by them?).
Reversing them with some online tools does not yield much results (yet ... might be fun to look into this at some point).
Next, In the Linphone client, click on Account Assistant and `Use a SIP account`, use your account login.
Put `username-hotmail.com-FEIOFJEIOFJEOIFJ` in `username`.
`XXXXXXX.bs.iotleg.com` in `domain.`
Replace with your values of course... from `/etc/flexisip/users/users.db.txt`
No need to enter password.
Transport `TCP`.
Next, create a new contact in the Linphone client, give it a name and enter `sip:c300x@XXXXXXXX.bs.iotleg.com` in `SIP account` field.
Next, we need to make sure we can reach `XXXXXXXX.bs.iotleg.com`. So add this hostname to the hosts file (guides are online).
Hint: edit /etc/hosts on Mac/Linux ... windows ... check on google ;-)
Make it resolve to the IP of your intercom unit.
Restart the Linphone client (sometimes it needs to reload the hostname from the hosts file you just added).
Next, tail the log on the intercom.
```
# tail -f /var/log/log_rotation.log
```
Start a video call to the `c300x` contact on Linphone client and check the log for all the useful info in there. I won't copy/paste much. It has sensitive data.
You should at least find:
```
flexisip - - - nta: received INVITE sip:c300x@XXXXXXX.bs.iotleg.com SIP/2.0 (CSeq 20)
flexisip - - - nta: INVITE (20) to message callback
flexisip - - - New SipEvent 0x16031b4 - msg 0x15fdf30
flexisip - - - Receiving new Request SIP message INVITE from sip:username-hotmail.com-OIEFJEIOFJEIOFJEOIFJEFOJ@XXXXXXX.bs.iotleg.com :#012INVITE sip:c300x@XXXXXXX.bs.iotleg.com SIP/2.0#015#012Via: SIP/2.0/TCP 192.168.XX.XX:57366;branch=z9hG4bK.m79GjLeBw;rport=57366#015#012From: <sip:username-hotmail.com-OIEFJEIOFJEIOFJEOIFJEFOJ@XXXXXXX.bs.iotleg.com>;tag=uK~Uw7yNa#015#012To: sip:c300x@XXXXXXX.bs.iotleg.com#015#012CSeq: 20 INVITE#015#012Call-ID: m1FJTLgmTM#015#012Max-Forwards: 70#015#012Supported: replaces, outbound, gruu#015#012Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, NOTIFY, MESSAGE, SUBSCRIBE, INFO, PRACK, UPDATE#015#012Content-Type: application/sdp#015#012Content-Length: 847#015#012Contact: <sip:username-hotmail.com-OIEFJEIOFJEIOFJEOIFJEFOJ@XXXXXXX.bs.iotleg.com;gr=urn:uuid:122beb47-575d-0013-9522-44f54cf1bc85>#015#012User-Agent: Linphone Desktop/4.4.1 LinphoneCore/5.1.19-1-g6cdd0918e#015#012#015#012v=0#015#012o=username-hotmail.com-OIEFJEIOFJEIOFJEOIFJEFOJ 2552 1319 IN IP4 192.168.XX.XX#015#012s=Talk#015#012c=IN IP4 192.168.XX.XX#015#012t=0 0#015#012a=rtcp-xr:rcvr-rtt=all:10000 stat-summary=loss,dup,jitt,TTL voip-metrics#015#012m=audio 7078 RTP/AVP 96 97 98 0 8 18 99 100 101#015#012a=rtpmap:96 opus/48000/2#015#012a=fmtp:96 useinbandfec=1#015#012a=rtpmap:97 speex/16000#015#012a=fmtp:97 vbr=on#015#012a=rtpmap:98 speex/8000#015#012a=fmtp:98 vbr=on#015#012a=fmtp:18 annexb=yes#015#012a=rtpmap:99 telephone-event/48000#015#012a=rtpmap:100 telephone-event/16000#015#012a=rtpmap:101 telephone-event/8000#015#012a=rtcp-fb:* trr-int 1000#015#012a=rtcp-fb:* ccm tmmbr#015#012m=video 9078 RTP/AVP 96 97#015#012a=rtpmap:96 VP8/90000#015#012a=rtpmap:97 H264/90000#015#012a=fmtp:97 profile-level-id=42801F#015#012a=rtcp-fb:* trr-int 1000#015#012a=rtcp-fb:* ccm tmmbr#015#012a=rtcp-fb:96 nack pli#015#012a=rtcp-fb:96 nack sli#015#012a=rtcp-fb:96 ack rpsi#015#012a=rtcp-fb:96 ccm fir#015#012a=rtcp-fb:97 nack pli#015#012a=rtcp-fb:97 ccm fir#015
...
Step: 1#011Found contact sip:c300x@2068164.bs.iotleg.com -> sip:c300x@127.0.0.1:55788;transport=tcp;regid=fze5fze5f usedAsRoute:0
Step: 1#011Returning collected records 1
flexisip - - - Listener created for sipUri = sip:c300x@XXXXXXXX.bs.iotleg.com
flexisip - - - Subscribe topic = c300x@XXXXXXXX.bs.iotleg.com
flexisip - - - Inject Request SIP message:#012INVITE sip:c300x@127.0.0.1:55788;transport=tcp ... rest ommitted
...
flexisip - - - Record route added.
flexisip - - - Sending Request SIP message to sip:c300x@127.0.0.1:55788;transport=tcp#012INVITE sip:c300x@127.0.0.1:55788;transport=tcp
...
flexisip - - - Message is sent through an outgoing transaction.
flexisip - - - nta: received 100 Trying for INVITE (20)
flexisip - - - nta: received 488 Not acceptable here for INVITE (20)
flexisip - - - nta: 488 Not acceptable here is going to a transaction
```
This is where the Linphone client hangs up with a message `Remote party cannot accept the call`.
We probably don't provide all the needed options for SIP ... or it is requiring TLS in someway maybe.
A normal call doesn't return a `488` but a `200` and continues like this where the `aswm` also kicks in.
```
flexisip - - - nta: sent 200 Ok for INVITE (20)
flexisip - - - Looking at fork contexts with key c300x@XXXXXXX.bs.iotleg.com
flexisip - - - Remove fork c300x@XXXXXXX.bs.iotleg.com from store
flexisip - - - Unsubscribe topic = c300x@XXXXXXX.bs.iotleg.com
flexisip - - - Destroy ForkCallContext 0xc43cd4
<aswm> - - - SIP: Media established
```
After some time on Google I came across this post:
https://stackoverflow.com/questions/15852013/sip-2-0-488-not-acceptable-here-error
And indeed comparing the normal call and the attempted called revealed in the SIP message:
```
... TTL voip-metrics#015#012a=DEVADDR:20#015#012m=audio 46160 RTP/SAVP 96 97 98 0 8 99 ....
and
... TTL voip-metrics#015#012m=audio 7078 RTP/AVP ...
```
Since everyone was talking about encryption I went to look for the `Encryption` option in the Linphone settings.
Under `calls and chat` change from `none` to `SRTP` under Encryption.
Call your contact again. Your intercom will ring. You can pickup with the green phone.
This was an intercom to Client connection. Not what I was expecting, but at least we managed to setup a call.
## Other findings
### Syncing of users
At a moment I had a `Forbidden` error when I tried connecting from the Door Entry App to the intercom.
It seemed like my users were gone in `/etc/flexisip/users/users.db.txt`.
When tracing `bt_answering_machine` I came across an interesting line:
```
$ strace -p 650 2>&1 | egrep -v "gettime|poll"
stat64("/var/tmp/user_list.txt", 0x7ea109f0) = -1 ENOENT (No such file or directory)
```
Every once in a while this file gets polled.
I placed an empty file there at some point to test what would happen.
I think this file was then copied to `/etc/flexisip/users/users.db.txt and erased my users.
We need to verify this.
Anyhow, I wasn't sure how to reset this. I deleted the Door Entry app and reinstalled it.
You then get a popup saying that it will `sync` and be available shortly.
This is a lovely time to check out the `/var/log/log_rotation.log` file to see some interesting stuff happening.
You will see how the SIP accounts/records/routes get created.
```
flexisip - - - Syncing password file
flexisip - - - Opening file /etc/flexisip/users/users.db.txt
flexisip - - - Syncing done
flexisip - - - Found registrar domain: XXXXXXXX.bs.iotleg.com
flexisip - - - Reading static records file
flexisip - - - Creating AOR alluser@XXXXXXXX.bs.iotleg.com association
flexisip - - - Trying to insert new contact sip:....omitted....
... more accounts being added ....
```
### Receive a "ring door" in the linphone client:
In order for the door call to be received also by the linphone client it is necessary to add the registered user in the following files (*respecting the already existing input format*):
```
/etc/flexisip/users/users.db.txt
/etc/flexisip/users/route.conf
/etc/flexisip/users/route_int.conf
```
### Config a Baresip client in Raspberry (comparable to the desktop client linphone)
the steps will then be divided into 2 main parts:
1) will be to set up c300x to accept the raspberry client
2) install and configure the Baresip client in raspberry
So...
1° part
1) a) create a new user in `/etc/flexisip/users/users.db.txt`
1) b) add raspberry ip address in `/home/bticino/cfg/flexisip.conf` separed with a blank space --> `trusted-hosts=`
1) c) add user created in 1)a) into `/etc/flexisip/users/route.conf` and `/etc/flexisip/users/route_int.conf`
1) d) reboot c300X
2° part on raspberry (this is based on the document https://github.com/spbroot/sipdoorbell rep)
2) a) Adding the ALSA Loopback Sound Device:
```
sudo su
echo 'snd-aloop' >> /etc/modules
exit
```
---
**NOTE**
On Ubuntu server you might need to install:
$ sudo apt install linux-modules-extra-$(uname -r) alsa-base
sudo reboot
To have the ALSA config
---
2) b) Making changes to the ALSA configuration
Create or edit the `/etc/asound.conf` file
```
# output device
pcm.loop_out {
type dmix
ipc_key 100001
slave.pcm "hw:Loopback,0,0"
}
# input device
pcm.loop_in {
type dsnoop
ipc_key 100002
slave.pcm "hw:Loopback,1,1"
}
# plug device
pcm.sipdoorbell_return {
type plug
slave.pcm 'loop_out'
hint {
show on
description 'sipdoorbell return channel'
}
}
# plug device
pcm.sipdoorbell_main {
type plug
slave.pcm loop_in
hint {
show on
description 'sipdoorbell main channel'
}
}
```
!!! if you get homebridge error when open doorbell device on iphone/ipad/AppleTV:
[error] cannot open audio device sipdoorbell_main (No such file or directory)
[error] sipdoorbell_main: Input/output error
or
[error] cannot open audio device sipdoorbell_return (No such file or directory)
[error] sipdoorbell_return: Input/output error
put this code at the end `/usr/share/alsa/alsa.conf` instead `/etc/asound.conf`
Taken from: https://github.com/spbroot/sipdoorbell
2) c) Installing Baresip
`sudo apt install baresip`
Run baresip to create a configuration file, after launch press 'q' to exit
Making changes to the baresip configuration file: file `/home/pi/.baresip/config`
Remove the comment from the following lines and make changes
```
sip_listen 0.0.0.0:5060
audio_player alsa,hw:Loopback,0,1
audio_source alsa,hw:Loopback,1,0
audio_alert alsa,hw:Loopback,0,1
ausrc_srate 48000
auplay_srate 48000
ausrc_channels 2
auplay_channels 2
module opus.so
module srtp.so
module httpd.so
http_listen 0.0.0.0:8000
```
2) d) Add a SIP account to the file `/home/pi/.baresip/accounts`
`<sip:*USERCREATEDIN1-a*@XXXXXXX.bs.iotleg.com;transport=tcp>;outbound="sip:*IPINTERCOM*;transport=tcp";auth_user=*FIRT_PART_USER_CREATED_IN_1a*;auth_pass=NOPASSNEED;mediaenc=srtp;audio_codecs=opus/48000/2;audio_source=alsa,default`
2) e) Reboot Raspberry
2) f) run Baresip:
```
baresip
```
Try to made a call with `/dial` command
### Processes
| Process | Function | More info |
|----------|-------------|-------------|
| BtClass | The frontend GUI on the intercom | Started from: /home/bticino/binBtClass_qws and loading it's UI from /home/bticino/bin/gui |
| bt_answering_machine | Handles recordings of incoming calls ||
| openserver | OpenWebNet implementation | https://developer.legrand.com/documentation/open-web-net-for-myhome/ listening on port 20000 see cat /var/tmp/conf.xml for the password under `pwd_open` - also handling the Authentication of Door Entry app at load (see Axiom client in logs) |
| bt_vct | Receives openwebnet commands| on port 30006, already used to unlock door |
### Commands
* Open and close lock `echo *8*19*20## |nc 0 30006; sleep 1; echo *8*20*20##|nc 0 30006` (also sometimes 21 instead of 20)
### Viewing firmware differences
This step of notes explains the commands you can use to compare the different firmware versions.
You will need to understand how to extract the firmware and mount it beforehand.
The instructions are for executed on a Linux host. You might need to be root to mount.
You also need git to be installed to make life easy.
```
root@server:/home/user/download# mount /home/user/1_7_17/btweb_only.ext4 /media/1_7_17
root@server:/home/user/download# mount /home/user/1_7_19/btweb_only.ext4 /media/1_7_19 && cd /media
root@server:/media# ls
1_7_17 1_7_19
root@server:/media# mkdir tmpdir
root@server:/media# cp -r 1_7_17/* tmpdir/ && cd tmpdir
root@server:/media/tmpdir# git init && git add * && git commit -m Initial
root@server:/media/tmpdir# cp -r ../1_7_19/* .
root@server:/media/tmpdir# git diff
```