@bltavares takeaways: - Franz hypercore's implementation works on top of AsyncRead/AsyncWrite, while colmeia is using Socket internally. It would be nice to also abstract on top of AsyncRead/AsyncWrite. - My first attempt to build the network layer was using libp2p (jan/2020), but I stopped as soon as I noticed I couldn't connect to bare dat.js clients - It was really simple to make libp2p-rs talk to another libp2p-rs process. - Main points that led me into looking for libp2p-rs implementation: - mdns already existing - dht already existing - later - [discussions of a ble discovery](https://github.com/libp2p/rust-libp2p/pull/971) - My interest - be able to run Dat on mobile apps. - eg: dat-installer but without embedding node - bltavares plans of using dat on mobile: - photo sharing app between android/ios for friends - pocket-like reading app - using unwalled.garden file format for storage, support for scientific papers like [sciencefair](http://sciencefair-app.com/), sharing libraries with friends, [extracting archive-view from posts (old dead project)](https://github.com/bltavares/artichoke/) - wishful - BLE support - wishful - browser support - wasm? - My first attempt had only tried to connect libp2p to dat1. dat1 does not use NOISE, but dat2 does, as do lib2p - colmeia, like libp2p, has a discovery layer and a connection layer. For example, [colmeia takes a stream of SocketAddr](https://github.com/bltavares/colmeia/blob/master/src/bin/colmeia-sync.rs#L25) for discovery. It could be extended to have many discovery mechanisms `dat.with_discovery(dat.lan().chain(dat.dht()).chain(libp2p.ble()))` - Except that libp2p discovery cannot be moved outside of the networks stack - QUIC protocol on libp2p also takes care of multiplexing natively. It seems like we could remove the multiplexing config form libp2p stack to use on dat, as it has it's own multiplexing. - It would be possible to write a `DatMdnsDiscovery` and `DatHypwerswarmDiscovery` for libp2p, but: - It is not possible to remove capability/protocol negotiation from libp2p connection handling. **This would not let libp2p datrs talk to node dat.js directly, as they would not negotiate the protocols properly (is this correct?)** - split the network efforts on node and rust if we can't find a way to bridge them - Could it be possible to design datrs hyperstorage to receive a network stack? So far, how rust works, porting led me to a path were `Dat` has a network stack "inside" of it, due to ownership of the connection socket. This is also driven by porting JS code and tring to keep things close. Trying out libp2p, the application seems to be written "inside" the stack. *(is this correct?)* The designs seems to not match if so. If possible to design the network stack to be plugable: - Would it possible to run `Dat` with two network stacks and act as a bridge between beaker and rust? - Otherwhise, we would need to have bridge/relay services, maybe sharing the underlying disk storage? fsync and fs-notifier felings. - Also, would it be worth to maintain a dual-stack project? - Specially as we are always falling behind the dat.js changes and evolutions, and most of the network effect is on moving into hyperswarm now (Beaker 2, dat-daemon). - I'm a hobbist, and I barely have time to push datrs into something usable so far. - Was there any attempts in the past to move dat.js to libp2p? That would make things much easier to interop and avoid 2 networks stacks across the network.