Prerequisites - HackMD

## [p2p-news-app](https://github.com/trulynotafan/p2p-news-ap) <div style="font-size: 0.5em"> This project is all about building a web-based peer-to-peer (P2P) "news reader" application. Think of it like a decentralized blog or a modern take on RSS/Atom news readers, but one that runs entirely without central servers. The main idea is straightforward: you, as a user, can publish your own news or blog posts directly to your personal feed. At the same time, you can subscribe to other people's feeds, pulling content from various sources. The app then brings all these subscriptions together into a single, personalized "news feed", so you can easily read everything from the blogs you follow. By using the powerful Hypercore/Hyperdrive P2P stack, the app makes sure content goes directly from one peer to another. This setup gives you a solid, server-less experience, putting you in charge of your content and subscriptions. It's a fully decentralized way to share and get information. </div> --- ## Dev Diary <div style="font-size: 0.7em"> **This presentation covers:** - Stack & Structure - Features & Functionality - Code & Progress - Learnings & Insight Different colors are used for each slide type: </div> <div style="font-size: 0.5em; color: #666; line-height: 1.5; margin-top: 20px;"> <span style="background:#141414; color: white; padding: 4px 8px; border-radius: 3px;">Technology</span> — Stack & concepts< <span style="background:#2E7D32; color: white; padding: 4px 8px; border-radius: 3px;">Project Overview</span> — Features & usage<br> <span style="background:#EF6C00; color: white; padding: 4px 8px; border-radius: 3px;">Code / Repos</span> — Commits & implementation<br> <span style="background:#6A1B9A; color: white; padding: 4px 8px; border-radius: 3px;">Worklogs</span> — Progress & issues<br> <span style="background:#8B4513; color: white; padding: 4px 8px; border-radius: 3px;">My Learnings</span> — Takeaways </div> --- # Prerequisites Use ↓ to navigate through the learning material. ---- ## Frontend - [Frontend Basics](https://youtube.com/playlist?list=PLubqJ-IuY27BQ7sDNZraRYcFWjmU3JABz) - [UI Principles](https://youtube.com/playlist?list=PLubqJ-IuY27BgNQnyNfwwjITBhQS6W6tC) ---- ## Project Management - [Task Management Guide](https://www.youtube.com/watch?v=xHcHnwgQQe0&list=PLubqJ-IuY27B7v7XikPxa6wFLELc8bqSD) Understand workflows and tracking progress. ---- # Contact Have you tried **[Keet](https://keet.io)**? <div style="font-size: 20px; line-height: 1.6; margin-top: 16px;"> - Keet is the official P2P messenger built in bare runtime. - If you get stuck on something **stack-related**, ask in the Pear community group. </div> <div style="margin-top: 30px; font-size: 14px; color: #ccc;"> Keet Invite Link: <code style="font-size: 14px;">pear://keet/yry7imzue1dj1jdno9qtoehfoff1839oiems3x7zkheychqq7jgof57wp46ohw4u1w63o6is6sax6j3swgdi66dmti8p8nfuyypfwewbuy</code> </div> --- # P2P Stack Overview  <div style="font-size: 20px;"> Core components powering the app: 1. Corestore + Hypercore 2. Hyperswarm 3. Hyperdrive 4. Protomux 5. DHT relay and hyper-webrtc Below is the list and description of all the modules used. </div> ---- ## Bare Runtime  <div style="font-size:20px; line-height:1.4;"> Bare is like Node.js — but stripped down to the essentials. It’s meant for small, lightweight environments — ideal for peer-to-peer apps that run in CLI or constrained systems. It gives just enough tooling (like filesystem, TCP sockets, etc.) to power your app without any bloat. Think of it like a racing car with only the parts needed to go fast. - [Bare GitHub](https://github.com/holepunchto/bare) - [Bare Docs](https://bare.pears.com) - [Holepunch Bare Intro](https://holepunch.to/blog/introducing-bare) </div> ---- ## Working with Bare runtime  <div style="font-size:20px; line-height:1.4;"> So far, had fun working with bare runtime, most of the native JS modules usually work perfectly fine, if they don't, you can always find bare alternatives on the official Holepunchto github. Also if anything with the modules go wrong you can always ask on keet (previously mentioned chatting app) on which people always help solving the problem. Also a big advantage of bare is it's a lot faster than nodejs and works on android aswell. Also would recommend to use the bare alternarive from github instead of using the native modules so you don't run into issues with native modules, which is mostly unlikely but still it's a risk. * **Porting Challenges:** My first task was porting a P2P demo to Bare. This involved tackling Node.js incompatibilities, like `sodium-native` which needed `sodium-universal` instead, or `process.uptime()` which I learned could be replaced with `process.hrtime()` from `bare-process` or simply a counter. * **Module Alternatives:** We also figured out we could replace the `derive-seed` module by using a helper function directly from the `corestore` module's codebase. </div> ---- ## Corestore + Hypercore  <div style="font-size:20px; line-height:1.4;"> Imagine each user has their own notebook (called a hypercore), where they can only write new pages — they can’t edit old ones. **Corestore** is like a library that organizes all these notebooks. When someone adds a new news post, it’s just a new page in their notebook. If another peer connects, they get a read-only copy of your notebook — verified and tamper-proof. This makes the system fully distributed, like everyone writing in their own logs, but able to read others’ logs live. * **Practical Use:** In the demo version made by serapath, peers would create their own hypercores ("books") and append simple data like an uptime counter. Other connected peers would then replicate these cores to read the updates. This helped me understand the append-only log and replication concepts. - [Hypercore Docs](https://docs.pears.com/building-blocks/hypercore) - [Corestore Intro (Dat Ecosystem)](https://hackmd.io/qjLOvrX0StiB5UIX9Xmqcg?view#Corestore--Hypercore) </div> ---- ## Hyperswarm  <div style="font-size:20px; line-height:1.4;"> Hyperswarm is what makes peers find each other. When a user runs the app, they join a common topic (like a virtual room). Anyone else in that same topic becomes a potential peer. It uses a Distributed Hash Table (DHT) behind the scenes — like a massive P2P phonebook — to discover peers and make encrypted, direct connections. No servers. No central control. Just swarm magic. - [Hyperswarm Docs](https://docs.pears.com/building-blocks/hyperswarm) - [Hyperswarm GitHub](https://github.com/holepunchto/hyperswarm) </div> ---- ## Working with hyperswarm  <div style="font-size:20px; line-height:1.4;"> Hyperswarm is a very straight-forward module, you can create a swam following the method in the docs and join the swarm with the unique topic, the topic could be a hex key or any other unique identity, and others can join the swarm using the same topic. We can use different techniques such as local bootstraps for speeding up the conenction process for development process and it helps a ton. We can join the swarm as a server or client or both, when other peers join, we can use promotomux (Explained below) to create a channel to start sending data such as text to other peers and recieve data from them. * **Hardcoded Topics:** To simplify connections, we used a hardcoded topic string (hashed to a discovery key) that all peers join. This way, they can find each other without manually exchanging keys. * **Local Bootstrappers:** For faster development and testing locally, setting up a local bootstrap server was super helpful to speed up peer discovery instead of relying solely on public ones. Some examples for local server are below: - [Create a P2P Test Network (bootstrapper.js)](https://github.com/datdotorg/datdot-service/blob/main/demo%2Fbootstrapper.js) - [Flamingo Wallet – app.js (lines 38–61)](https://github.com/ninabreznik/flamingo-wallet/blob/main/app.js#L38-L61) - [Flamingo Wallet – foo.js (lines 40–55)](https://github.com/ninabreznik/flamingo-wallet/blob/main/foo.js#L40-L55) - [Flamingo Wallet – mux-proto.js](https://github.com/ninabreznik/flamingo-wallet/blob/main/lib/mux-proto.js) </div> ---- ## Connecting to Web Peers  <div style="font-size: 20px;"> Getting our P2P app to work in browsers required some extra steps because browsers can't do raw P2P networking like command-line apps can. </div> ---- ### Hyperswarm DHT Relay  <div style="font-size:20px; line-height:1.4;"> This module is key for web peers. Since browsers can't directly speak the hyperswarm protocol (which often uses UDP), the DHT relay acts as a bridge. * **How it Works:** You run a relay server, Browser peers connect to this relay server using WebSockets. The relay then talks to the main hyperswarm DHT on behalf of the browser peers, forwarding messages back and forth. * **My Experience:** Setting this up involved running the relay script which I ran in nodejs cause the module had some problems with bare. We will look into that as it's not a big of deal at the moment. - [Hyperswarm DHT Relay GitHub](https://github.com/holepunchto/hyperswarm-dht-relay) </div> ---- ### Hyper WebRTC (The Next Step)  <div style="font-size:20px; line-height:1.4;"> While the relay gets browser peers connected, all their traffic still goes through that relay server. To make it more P2P and reduce load on the relay, the idea is to use `hyper-webrtc`. * **The Goal:** When two *browser* peers discover each other (via the relay), they can then try to establish a direct WebRTC connection between themselves, bypassing the relay for their actual data exchange. * **Current Status:** I've looked into this, but ran into some compatibility issues getting `hyper-webrtc` and its dependencies (like `werift`) to work smoothly with the Bare runtime environment or the browser bundling setup. This is something we will revisit. - [Hyper WebRTC GitHub](https://github.com/LuKks/hyper-webrtc) </div> ---- ## Helper Modules  <div style="font-size: 20px;"> A few other important pieces in our toolkit. - 00. streamx - 01. hyperswarm - 02. corestore - 03. hypercore - 04. hyperbee - 05. sodium-universal & hypercore-crypto - 06. b4a - 07. hyperdrive - 08. protomux - 09. compact-encoding - 10. keet-identity-key - 11. bip39-mnemonic We will further add modules as the project continues. ---- ## b4a  <div style="font-size:20px; line-height:1.4;"> b4a is a library for working with raw binary buffers and it is compatible with brower ArrayBuffers. It provides a set of functions for bridging the gap between the Node.js Buffer class and the Uint8Array class. Basically you dont have to worry about whether you're dealing with buffers or typed arrays. [b4a GitHub](https://github.com/holepunchto/b4a) </div> ---- ## Sodium  <div style="font-size:20px; line-height:1.4;"> Sodium is a well-audited and highly secure cryptographic library that provides a wide range of cryptographic primitives, including encryption, digital signatures, secure random number generation, and digital fingerprinting (i.e., hashing). We use `sodium-universal` which aims to work in both Node.js and browser environments. [sodium-universal GitHub](https://github.com/holepunchto/sodium-universal) </div> ---- ## Derive seed & Crypto  <div style="font-size:20px; line-height:1.4;"> These are some other cryptographic helper functions to make things secure and private. They allow us to generate unique keys and secure seeds for things like encrypted communication, secure logins, and safe data handling. * **Deriving Keys:** A key concept was learning to derive multiple cryptographic keys (for different "books" or purposes) from a single master seed. This simplifies key management. We ended up using a function from `corestore`'s own code for this after finding `derive-seed` module was a bit old. [Bare Crypto GitHub (for general crypto needs in Bare)](https://github.com/holepunchto/bare-crypto) </div> ---- ## Protomux  <div style="font-size:20px; line-height:1.4;"> Protomux is used for multiplexing multiple message oriented protocols over a stream. Basically, Protomux lets you send multiple well-structured messages (not just raw binary buffers) over a single stream, such as raw UDP or WebSocket. Each message can follow its own mini-protocol, making it easier to organize communication without opening separate connections. [Protomux GitHub](https://github.com/holepunchto/protomux) [Compact Encoding GitHub](https://github.com/holepunchto/compact-encoding) </div> ---- ## streamx  <div style="font-size:20px; line-height:1.4;"> streamx is an enhanced, modern reimplementation of Node.js core streams designed for better lifecycle management, simplified APIs, and improved error handling. It supports both binary and object modes natively, integrates cleanly with `AbortSignal`, and includes streamlined `pipe()` behavior with built-in backpressure and destruction handling. With a smaller browser footprint and full compatibility where reasonable, `streamx` offers a lightweight, developer-friendly alternative for building fast, reliable streaming logic in Node.js and the browser. [streamx GitHub](https://github.com/mafintosh/streamx) </div> ---- ## hyperbee  <div style="font-size:20px; line-height:1.4;"> Hyperbee is an append only B-tree based on Hypercore. It provides a key/value-store API, with methods for inserting and getting key-value pairs, atomic batch insertions, and creating sorted iterators. It uses a single Hypercore for storage, using a technique called embedded indexing. It provides features like cache warmup extension, efficient diffing, version control, sorted iteration, and sparse downloading. [hyperbee GitHub](https://github.com/holepunchto/hyperbee) </div> ---- ## hyperdrive  <div style="font-size:20px; line-height:1.4;"> Hyperdrive is a secure, real-time distributed file system designed for easy P2P file sharing. We use it extensively inside Holepunch; apps like Keet are distributed to users as Hyperdrives, as is the Holepunch platform itself. [hyperdrive GitHub](https://github.com/holepunchto/hyperdrive) </div> ---- ## keet-identity-key  <div style="font-size:20px; line-height:1.4;"> keet-identity-key is a tool that helps create secure digital keys from a secret phrase, known as a mnemonic. These keys follow a strong security method called Ed25519. It lets you safely connect new devices by proving they belong to you. One device can prove that another device is trusted, and others can check if that proof is real. This system is useful for apps like Keet, where there's no central server and everything needs to stay secure and private across multiple devices. [keet-identity-key GitHub](https://github.com/holepunchto/keet-identity-key) </div> ---- ## bip39-mnemonic  <div style="font-size:20px; line-height:1.4;"> Used for generating human-readable mnemonic phrases (like wallet seed phrases) from binary seeds and vice versa. Useful for account recovery and secure seed storage. [bip39 GitHub](https://github.com/holepunchto/bip39-mnemonic) </div> ---- ## compact-encoing  <div style="font-size:20px; line-height:1.4;"> A lightweight, fast JavaScript library for encoding and decoding various data types into buffers. It’s designed for building small, efficient parsers and serializers — especially useful in networking, storage, or binary protocols. [bip39 GitHub](https://github.com/holepunchto/bip39-mnemonic) </div> --- ## Worklogs  <div style="font-size:20px;"> Slides below contain worklogs from the beginning to what we have acheived so far in our [p2p-news-app](https://github.com/trulynotafan/p2p-news-app). </div> ---- ## Worklog [#1](https://github.com/trulynotafan/p2p-news-app/issues/1) - Initializing repository  <div style="font-size:20px;"> In the first worklog, we initizalized the repository, added a simple index.html and index.js, added budo in our dependecy as it provides live updates of the site as we work on our project. We also created basic structure of this HackMd document. The link of the repo can be found [here](https://github.com/trulynotafan/p2p-news-app) ![Capture](https://hackmd.io/_uploads/H1GLTnEZxl.png) </div> ----  <div style="font-size:20px;"> Generally we just added a basic canvas that we will be using for the initialization of our project. The videos mentioned in this worklog are the videos which I've preveiously mentioned in the Prerequisites slide on how we do frontend and project managment. As listed, next step is to further refine this hackmd document. ![Capture](https://hackmd.io/_uploads/S1P-A2Vbll.png) </div> ---- ## Worklog [#2](https://github.com/trulynotafan/p2p-news-app/issues/2) - Refining Document  <div style="font-size:20px;"> In the second worklog, we further refined this dev diary by adding proper lists and brief introduction of each module that we are using. We also created a step by step guide to p2p demo created by serapath. ![Capture](https://hackmd.io/_uploads/H1aoQJSWlx.png) </div> ----  <div style="font-size:20px;"> Basically the main thing in this worklog was to add a good step by step p2p demo guide so users could understand the apis and functions of different modules and how they worked. You can now find the p2p demo slide on the right of the worklogs to understand the core concpets. </div> ----  <div style="font-size:20px;"> # Worklog [#3](https://github.com/trulynotafan/p2p-news-app/issues/3) - implement p2p-news-app ![Screenshot from 2025-06-01 09-48-19](https://hackmd.io/_uploads/S1Wpi8FGgg.png) </div> ----  <div style="font-size:20px;"> In the third work log, I first reviewed all the previous playlists related to task management. We then added basic P2P files to the repository, which will be further edited later. Next, we took the basic P2P demo created by Serapath and converted it to work with Bare. The P2P demo is explained in the right-hand slides. </div> ----  <div style="font-size:20px;"> ![image](https://hackmd.io/_uploads/r1pqhLYzlx.png) This issue serves as the root issue for the P2P news app. Whenever we add a new issue or task, we will first document it here and then continue working on the specific sub-issue. As you can see in this issue, the next step is to add the protomux feed key — that is, the core key exchange required for peers to replicate the book keys. Since the sub-issue will involve more details, we created a new issue and linked it to this parent issue. This parent issue will only be closed once the full P2P news app is implemented, as indicated by the planned output: `@output 🏁❓️ p2p-news-app repository` </div> ----  <div style="font-size:20px;"> # Worklog [#4](https://github.com/trulynotafan/p2p-news-app/issues/4) - Protomux feedkey and identity exchange **Why This Issue Was Needed:** The basic P2P demo only allowed simple data replication. To build a proper news app, peers needed to exchange their identities and feed keys so they could discover and replicate each other's content. This required implementing a protocol layer using Protomux. **The Problem:** Peers could connect but couldn't properly identify each other or exchange the keys needed for content replication. Without this, the app would just be isolated peers with no way to share or discover content. **Solution Implemented:** - Created Protomux channels for identity and feed key exchange - Added connection upgrade logic for browser peers using WebRTC - Restructured repository to separate native and web peer logic - Created helper modules for shared protocol functionality </div> ----  <div style="font-size:20px;"> # Worklog [#4] - Screenshot ![image](https://hackmd.io/_uploads/S1PU8FaHgx.png) </div> ----  <div style="font-size:20px;"> ![image](https://hackmd.io/_uploads/H1ec8t6See.png) </div> ----  <div style="font-size:20px;"> # Worklog [#4] Continued - Repository Structure **Why Restructuring Was Needed:** The original code had everything mixed together, making it hard to maintain and understand. We needed clear separation between native and web peers while sharing common functionality. **Key Changes:** - Moved all main files to `src/node_modules/filename/index.js` - Created separate helper function files for shared code - Added `bin/cli.js` for native peer entry point - Added `web/page.js` for web peer entry point - Implemented standardx for consistent code formatting **Impact:** This structure made the codebase much more maintainable and allowed for better code reuse between different peer types. </div> ----  <div style="font-size:20px;"> # Worklog [#4] Structure - Screenshot ![image](https://hackmd.io/_uploads/ByFJvKpHxe.png) </div> ----  <div style="font-size:20px;"> # Worklog [#5](https://github.com/trulynotafan/p2p-news-app/issues/5) - Slides Updates and Module Boundaries **Why This Issue Was Needed:** As the project grew, we needed to document our progress and improve code organization. The existing slides were outdated, and the code had duplicate logic that needed to be consolidated. **The Problem:** - Documentation was falling behind the actual development - Code had duplicate protocol logic between native and web peers - Variable naming was inconsistent (mixing camelCase and snake_case) - Module boundaries were unclear, making code hard to reuse **Solution Implemented:** - Updated HackMD slides with latest worklogs and learning process - Created protocol helper module to eliminate code duplication - Standardized all code to use snake_case naming - Improved module boundaries for better reusability </div> ----  <div style="font-size:20px;"> # Worklog [#5] - Screenshot ![image](https://hackmd.io/_uploads/HysB_FTrgl.png) </div> ----  <div style="font-size:20px;"> # Worklog [#6](https://github.com/trulynotafan/p2p-news-app/issues/6) - Add bip39-mnemonic and production relay support **Why This Issue Was Needed:** The app needed a way to generate consistent identities from human-readable phrases (like wallet seed phrases) and required a production relay for browser peers to work in real-world scenarios. **The Problem:** - No way to derive consistent keys from mnemonic phrases - Browser peers only worked locally, not in production - Cryptographic functions weren't compatible across different environments - No persistent identity management **Solution Implemented:** - Added bip39-mnemonic support for generating keys from seed phrases - Deployed production relay on Railway for browser peer connectivity - Fixed cryptographic compatibility issues between browser and native environments - Created environment detection to use local vs production relay </div> ----  <div style="font-size:20px;"> # Worklog [#6] - Screenshot ![image](https://hackmd.io/_uploads/HkqcuKTHll.png) </div> ----  <div style="font-size:20px;"> # Worklog [#7](https://github.com/trulynotafan/p2p-news-app/issues/7) - Use keet-identity-key for pairing multiple devices **Why This Issue Was Needed:** Users should be able to use the same identity across multiple devices (phone, laptop, etc.) just like modern apps. This requires secure device pairing and identity management. **The Problem:** - Each device had a completely separate identity - No way to sync content or settings across devices - Users couldn't access their data from different devices - No secure way to prove device ownership **Possible Solution** - Add multi-device pairing for native peers using keet-identity-key - Implement identity pairing via autobase - Create helpers for identity management </div> ----  <div style="font-size:20px;"> # Worklog [#7] - Screenshot ![image](https://hackmd.io/_uploads/HkakYFpBlg.png) </div> ----  <div style="font-size:20px;"> # Worklog [#7] Feedback and Technical Issues **Feedback Received:** The implementation worked but needed improvements in several areas: - Code style consistency (use Bare.argv instead of process) - JavaScript formatting issues - Duplicate message handling for paired devices - Need for better device naming and identity management **Technical Issues Identified:** - Peers receiving duplicate "Hello, peer!" messages from identical peers. - Need for cross-device-name system for every identity - Joining devices should adopt the official name of the autobase identity - Some code formatting and dependency issues **Summary** Currently skipped this, because we were haing issue trying to implement autobase in browsers, we will continue it in the future. </div> ----  <div style="font-size:20px;"> # Worklog [#8](https://github.com/trulynotafan/p2p-news-app/issues/8) - Blogs UI, persistent storage, Blog handling **Why This Issue Was Needed:** The core P2P infrastructure was working, but we needed the actual news/blog functionality. Users needed a way to publish, subscribe to, and read blog posts in a decentralized manner. **The Problem:** - No user interface for creating or reading content - No persistent storage for blog posts - No way to subscribe to other users' feeds - No content replication between peers **Solution Implemented:** - Created minimal blog UI for publishing, subscribing, and viewing posts - Implemented blog publishing logic using Hyperdrive and Hypercore - Added peer discovery and replication of published blogs - Created shared blog helper module for both native and web peers - Built persistent storage system for subscriptions and content </div> ----  <div style="font-size:20px;"> # Worklog [#8] - Screenshot ![image](https://hackmd.io/_uploads/ryFmcYargx.png) </div> ----  <div style="font-size:20px;"> # Worklog [#8] Technical Architecture **How the Blog System Works:** - Each peer creates a Hyperdrive for their blog content - Blog posts are stored in Hyperdrive, metadata in Hypercore - When peers connect, they exchange feed keys via Protomux - Subscriptions are saved locally (localStorage for browser, JSON for native) - Two separate cores: one for own posts, one for replicated content - When someone subscribes, only your blog posts are shared (not replicated content) **Technical Challenges Overcome:** - Built browser-compatible version of truncate function for Random-access-web - Created patches for corestore modules to work in browsers </div> ----  <div style="font-size:20px;"> # Worklog [#8] Current Status and Future Direction **What's Working:** - Complete blog publishing and replication system - Cross-platform compatibility (native and web peers) - Persistent storage and subscription management - Real-time content synchronization between peers - Shared codebase between different peer types **Current Limitations:** - No custom relay configuration - Some deployment issues with GitHub Pages/Vercel - Code could be more concise in some areas **Next Steps:** - Add custom relay input in configuration - Complete documentation updates - Investigate deployment issues </div> --- ## P2P Demo  <div style="font-size:20px;"> Before we continue, let's understand how p2p works using a basic [p2p-demo](https://github.com/wizardamigos/p2p-demo) created by [Serapath](https://github.com/serapath). Below is the full description of the demo and how p2p works. </div> ---- ## The structure:  <div style="font-size:20px;"> The demo contains 3 files. 1. **Package.json** which contains the dependencies mentioned previously. 2. **writer-peer.js** which acts as a server peer and provides a peer key as well as a data replication core key. 3. **reader-peer.js** which acts as a client peer and uses the core key provided by the writer-peer to join the swarm and replicate the data (which in this case is the uptime). </div> ---- ## Usage  <div style="font-size:20px;"> The writer-peer can be started using **node writer-peer.js**![Capture](https://hackmd.io/_uploads/ryjN_pV-ex.png) </div> ----  <div style="font-size:20px;"> You can then copy the core key from the writer-peer to be used for the reader-peer. Then we start the reader-peer and paste the core-key and provide it a name: ![Capture](https://hackmd.io/_uploads/BkJROT4bxg.png) </div> ----  <div style="font-size:20px;"> After the connection is made, the peers start replicating each others append time, and store it in thier storage as append only logs. The data is also logged in the console. ![Capture](https://hackmd.io/_uploads/Bkv8iTV-xl.png) </div> ---- ## How it works  <div style="font-size:20px;"> Now that we have looked at its working, lets understand how it works. Lets start from the code of writer-peer.js. ![Capture](https://hackmd.io/_uploads/rkujjTVWxx.png) The first part of the code is to import all the required modules. 1. b4a is buffer for array, to work with typed arrays and we dont have to worry about browser compatiblity. 2. corestore, the library that provides the append only logs books called hypercores. 3. Hyperswarm, uses dht to make peers find each other. 4. Sodium, a cryptographic library that provides different cryptographic functions. 5. derive seed and crypto are also cryptography helper functions. These modules are already discussed with detail in previous prerequisites slides. </div> ----  <div style="font-size:20px;"> After importing the modules, the start() function, initiates the proess.![Capture](https://hackmd.io/_uploads/SJWDRTV-lx.png) After initialization, we create a label named "writer", wrapped in terminal color codes to make the word appear green in logs. Then we create an opts object with three properties: namespace: A unique label ('noisekeys') used for deriving the seed. seed: A random 32-byte value generated using crypto.randomBytes(32), acting as base entropy. name: A second-level label ('noise') to help derive a unique keypair. This opts object is used to generate a deterministic keypair for identity in the peer-to-peer network. We pass this opts object as a paramter to the create_noise_keypair function explained below. </div> ----  <div style="font-size:20px;"> ![Capture](https://hackmd.io/_uploads/Bys5gCNWgx.png) The function create_noise_keypair takes the opts object as parameter, Uses the derive_seed function which is predefined in the derive_key module to derive a unique seed using the three attributes of the opts object. This noiseSeed is then passed to sodium.crypto_sign_seed_keypair to generate a deterministic public/private keypair. The purpose of noiseSeed is to act as the secret input for creating a consistent identity—ensuring that the same keypair (and thus the same peer identity) is generated every time the same opts values are used. The function also creates two things: publicKey: This is like your username that others can see. It’s used to identify you in the network. secretKey: This is like your password. It’s kept private and used to prove that you really are who you say you are. Both are empty at first (just buffers), and then filled using the noiseSeed. </div> ----  <div style="font-size:20px;"> You can also use this function instead of the derive_key module as its old now: ![Capture](https://hackmd.io/_uploads/BJNAf0NZll.png) The deriveSeed function takes three things: primaryKey, namespace, and name. It: 1. Converts them into buffers (a format computers use to handle raw data). 2. Mixes them together using a secure hashing method. 3. Returns a 32-byte secret value (the **derived seed**) that can be used to create a unique and repeatable keypair (your identity). </div> ----  <div style="font-size:20px;"> After returning the public and private key, the public key is logged and a keypair object is created with these two keys. ![Capture](https://hackmd.io/_uploads/rk68NR4Wgx.png) After that, a new store is created like a storage or a book where we can save append-only logs (data that can only be added, not changed or deleted). This is where the uptime will be stored for other peers to replicate. Next we Construct a new Hyperswarm instance. In the swarm function we can pass: keyPair(which we are in this case): A Noise keypair that will be used to listen/connect on the DHT. Defaults to a new key pair. seed: A unique, 32-byte, random seed that can be used to deterministically generate the key pair. maxPeers: The maximum number of peer connections to allow. </div> ----  <div style="font-size:20px;"> When swarm is created we listen for another peer connection using swarm.on function ![Capture](https://hackmd.io/_uploads/HJPs8AN-xx.png) When a new peer joins the onnconnection funcion is called: ![Capture](https://hackmd.io/_uploads/r1cbwCNbxx.png) This function takes the socket and info from the swarm.on function, logs the public key, starts syncing the data between peers using the store and starts the uptime timer that calls the function append_more every second. </div> ----  <div style="font-size:20px;"> The function append_more uses process.uptime to find the uptime of the script and stores it in the time variable, then stamp variable uses the time and makes it a bit more fancy and easy to read. Then this time stamp is appened into our "book" using core.append() function. This is what other peers will replicate. ![Capture](https://hackmd.io/_uploads/SJhzF0E-ex.png) </div> ----  <div style="font-size:20px;"> After the swarm.on() function we create a new core form the store using store.get, give it a name and wait for the core to become ready. After that we log the corekey (the core data replication key) to the user for copying. ![Capture](https://hackmd.io/_uploads/H133c0EZel.png) We then join the discovey key using swarm.join(), We can also use 32bit hex topic, or hash a string to use it for joining the swarm. The discovery.flushed waits for the peer to be registered on the DHT before we start the replication of the cores. </div> ----  <div style="font-size:20px;"> And that's basically how the writer-peer works. Most of the code is just built in functions and APIs, we just have to know how to use these APIs. reader-peer is mostly the same, different stuff is discussed below. </div> ----  <div style="font-size:20px;"> 1. We use parse(process.argv.slice(2)) to get command-line flags such as the --corekey flag and the --name flag. ![Capture](https://hackmd.io/_uploads/HkEmARNbee.png) 2. The parse function coverts arguments into objects. ![Capture](https://hackmd.io/_uploads/rJS_ACVbex.png) 3. The validate() function checks these required flags and throws errors if missing. </div> ----  <div style="font-size:20px;"> 4. Instead of creating a new core, the reader loads (clones) an existing core using the given corekey. ![Capture](https://hackmd.io/_uploads/rJc11yH-gx.png) 5. The selection statement sees if there was any previous data, clones and logs it. 6. The onappend callback first trigger, downloads all existing entries. And then logs any newly appended data as it arrives. ![Capture](https://hackmd.io/_uploads/rJfmeyS-le.png) </div> ----  <div style="font-size:20px;"> 7. It joins the swarm as cient because the reader's job is to download data, not serve it. 8. Lastly, we can run multiple reader peers because they all join the same swarm using the same core key, but each has a different storage folder based on the unique --name argument. </div> ----  <div style="font-size:20px;"> In the end, this demo acts a simple but powerful p2p communication system. The writer peer appends messages to a log, and reader peers connect to read and sync those messages in real-time. All peers join the same discovery swarm using the corekey, and encryption is handled with noise keypairs for secure communication. Thanks to Hypercore, Corestore, and Hyperswarm, everything stays decentralized, fast, and efficient with no central server needed. If you want personally experiment with demo itself here's the [link](https://github.com/wizardamigos/p2p-demo/). </div> --- # Project Overview  <div style="font-size:20px;"> Now the overview of the project and what it does. **What it does:** - Create and publish blog posts - Discover other peers on the network - Subscribe to other blogs and sync their posts - Use both web browser and CLI interfaces - Store all data locally (no central server) **Key Features:** - Realtime synchronization - Cross-platform - Persistent subscriptions - Secure p2p communication </div> ---- ## Architecture Overview  <div style="font-size:20px;"> We have divided the code into different modules for easy uderstanding and reusability **Core Components:** - **Native Peer** (`bare-peer/index.js`) - CLI interface - **Web Peer** (`web-peer/index.js`) - Browser interface - **Blog Helpers** (`blog-helpers/index.js`) - blogs logic - **Protocol Helpers** (`protocol-helpers/index.js`) - For intial communication - **Crypto Helpers** (`crypto-helpers/index.js`) - Key management **Data Storage:** - **Browser**: IndexedDB + localStorage - **CLI**: File system + JSON files for metadata **Networking:** - Hyperswarm for peer discovery - WebRTC for browser-to-browser connections - DHT relay for browser-to-native communication - Protomux for protocol negotiation </div> ---- ## Blog System Architecture  <div style="font-size:20px;"> **How Blog Posts Work:** - Each user creates a Hyperdrive for their blog content - Blog posts stored in Hyperdrive, metadata in Hypercore - When peers connect, they exchange feed keys via Protomux - Subscriptions saved locally (localStorage for browser, JSON for native) - Two separate cores: one for own posts, one for replicated content **Content Flow:** 1. User creates post → stored in local Hyperdrive 2. Metadata appended to personal Hypercore 3. Other peers discover your feed key 4. Subscribers download and replicate your content 5. Realtime updates when new posts are published </div> ---- ## Peer Discovery & Connection  <div style="font-size:20px;"> **Discovery Process:** - All peers join the same hardcoded topic (hashed discovery key) - Hyperswarm uses DHT to find other peers **Connection Types:** - **Native-to-Native**: Direct connections - **Browser-to-Browser**: WebRTC direct connections - **Browser-to-Native**: Via DHT relay **Identity Exchange:** - Protomux channels handle protocol negotiation - Feed keys exchanged for content replication - Device information shared for identification - Secure keypair generation from mnemonic phrases </div> ---- ## Subscription Management  <div style="font-size:20px;"> **How Subscriptions Work:** - Users can subdscrube to available blogs on the network. **Storage Locations:** - **Web Interface**: `localStorage['subscribed_peers']` (JSON array) - **CLI Interface**: `./subscribed_peers.json` file **Subscription Process:** 1. Discover peer's blog feed key 2. Download all existing posts from their Hyperdrive 3. Set up realtime listeners for new posts 4. Store subscription locally for persistence 5. Auto-reconnect when peer comes back online **Content Organization:** - Personal posts kept separate from subscribed content </div> ---- ## Data Persistence & Storage  <div style="font-size:20px;"> **Cross-Platform Storage Strategy:** **Browser Environment:** - **IndexedDB**: Hypercore/Hyperdrive data storage - **localStorage**: Subscription metadata and settings **CLI Environment:** - **File System**: All data stored in local directories - **JSON Files**: Configuration and subscription metadata </div> ---- ## Web Interface Features  <div style="font-size:20px;"> **News Feed:** - Combined view of all subscribed blogs - Real-time updates when new posts arrive **My Blog:** - Personal posts only **Explore:** - Discover available blogs on the network - Subscribe/unsubscribe with one click **Post Creation:** - Simple form for title and content - Automatic UI updates **Config:** - Username and connection status - Custom relay configuration - Data reset functionality </div> ---- ## CLI Interface Features  <div style="font-size:20px;"> **Fucntionality is the same as web peer but we use a simple interactive menu** ![464830612-ab4b0ce6-f41f-47e2-b63c-59f806552650](https://hackmd.io/_uploads/HkuDSARBeg.png) </div> ---- ## Usage Examples  <div style="font-size:20px;"> **Getting Started:** **1. Start the Relay Server:** ```bash npm run relay # Relay running on ws://localhost:8080 ``` **2. Run Web Interface:** ```bash npm run build && npm run web # Server running at http://localhost:9966/ ``` **3. Run CLI Interface:** ```bash npx native-peer --name myusername # Native peer CLI started successfully ``` **4. Connect and Interact:** - Enter username and join network - Discover other peers automatically - Further interact via buttons or choices for CLI peers. </div> ---- ## Conclusion  <div style="font-size:20px;"> **Project Achievement:** Successfully built a **fully functional decentralized blog/news system** that demonstrates the power of P2P technology for content sharing and distribution. For more docs and code you can check the [repository](https://github.com/trulynotafan/p2p-news-app). </div> --- # My Development Journey & Learnings  Things I've learned througout my journey: ---- ## Coding Standards & Philosophy  <div style="font-size:20px; line-height:1.4;"> Working with Serapath, I've adopted a clear philosophy for building projects: * **Minimalism:** Use as little code as possible to achieve the goal. Smaller codebases lead to smaller apps, which are easier to maintain and less prone to errors from unrelated parts. * **Performance & Reliability:** Less code often means better performance and simpler auditing. * **Dependency Vetting:** Any third-party dependencies must be carefully evaluated — who maintains them, are they actively supported, and can they be relied upon in the future? * **Coding Style:** * We use **standardjs** for formatting (2-space indentation, no semicolons). * Variables and functions use **snake_case** instead of camelCase. * We avoid **class** and **this** in application code, preferring plain functions and variables. * **Vanilla JavaScript:** Focus on core JavaScript and P2P modules without relying on large frontend frameworks. This approach ensures our code is clean, efficient, and maintainable. </div> ---- ## Learning to use AI Tools  <div style="font-size:20px; line-height:1.4;"> Using AI (like ChatGPT) is now part of my workflow, but it took learning: * **Vibe Coding:** Early over-reliance on AI caused hallucinated code, style issues, and bugs * **Better Control:** Now AI is a targeted assistant for: * Small helper functions * Boilerplate or concept explanation * Following strict prompts for style and conciseness * Always review and integrate AI output manually * I must understand and control the code, not AI <strong>Prompt Template I use:</strong> <pre style="background:#8B4513; color:#eee; padding:30px; border-radius:10px;"> INSTRUCTIONS: Please be as minimal, short and concise as you can when you answer. Also, if code is involved, write all code in standardjs style, which includes 2 space indentation, no semicolons at line ends, but use snake_case naming for all references instead of camelCase. Only necessary changes should be made to any provided code and any added or changed line should be marked with a trailing "// CHANGE" comment. If lines are to be deleted, they should be replaced with a comment "// DELETED". QUERY: ... My actual prompt for the LLM ... </pre> </div> ---- ## Project Management & Workflow  <div style="font-size:20px;"> We're using a structured approach to manage the project: * **GitHub Issues for Tasks:** All work is broken down into tasks defined in GitHub issues. * **Worklogs:** For each work session, I create a worklog comment on the issue. This includes: * Which tasks were worked on and how long they took. * Links to outputs (like code commits or updated documentation). * Feedback on the process or any challenges. * Proposals for the next tasks. * This helps keep track of progress, especially with breaks, and makes sure we're aligned on what needs to be done. </div> ----  <div style="font-size:20px;"> ## My Programming and Learning Process This whole journey of working with the P2P stack has been a unique experience — definitely different from anything I’ve done before. What makes it fun is that there are so many different modules to try out, and a lot of the time, you get to build stuff in your own way instead of just following some strict tutorial or framework. But obviously, along with the fun, there have been quite a few moments of frustration, confusion, and hitting dead ends. So I wanted to write down some of the things I’ve learned, the struggles I’ve had, and how my coding process has changed along the way. First off, I don’t consider myself a fast learner. Most of what I’ve picked up over the years — whether it’s JavaScript, Node.js, or anything else — has come from YouTube tutorials, structured roadmaps, or just following along with what someone else already figured out. I’m not the kind of guy who just picks up docs and magically understands everything in one go. That’s why diving into the P2P stack was different for me. There aren’t as many complete roadmaps or tutorial series for this stuff. Most of it is documentation-based — and yeah, the docs are surprisingly good. Like, they’re written in a way that’s pretty straightforward, and honestly, copy-pasting code from the docs works more often than you’d expect. </div> ----  <div style="font-size:20px;"> But when it doesn’t work, it really doesn’t. Sometimes you hit a bug or something doesn’t behave like it’s supposed to, and there’s not enough context in the docs to figure out what the hell is going wrong. That’s where I started to really appreciate the community — especially the folks on [keet.io](https://keet.io). People there aren’t just helpful, they’re **crazy helpful**. Some of them have actually written these modules, and they’re just hanging out, ready to look at your code, debug with you, and explain stuff that would’ve taken you days to figure out on your own. I’ve had people go through huge chunks of my codebase just to find one little bug. It’s wild. And honestly, when you’re working with a lesser-known stack, having that kind of access to real devs (not influencers, not tutorial guys, but **actual module authors**) is such a blessing. One cool thing about this stack is how explicit and low-level everything is. You’re not abstracted away from the process. If something breaks, you can usually trace it back, step by step, and figure out what’s going wrong. The error messages are often very detailed, and once you get used to reading them, you can debug a lot of stuff without even asking anyone. That’s helped me a ton — like, not just fixing stuff, but actually understanding how things work under the hood. Now, if I had to complain about something, it’s the lack of real-world projects using this stack. I tried searching GitHub and random forums for actual apps or examples built using this ecosystem, and honestly, I didn’t find much. Just a handful of projects that clearly use the full stack and show you how to structure everything properly. That kind of sucks because when your code doesn’t work, it’s super useful to just peek at someone else’s implementation, compare it, and go “oh, that’s what I missed.” Without that, you're kind of flying blind. Luckily, the community again helps fill this gap — they often link working repos or help you reason about what you’re doing wrong. </div> ----  <div style="font-size:20px;"> Another major change in my process has been around using LLMs. Before this, I used to rely a lot on AI tools like ChatGPT, Claude, Gemini, etc., especially because I was working with stuff like Node.js/Discord.js or Python — you know, popular tech that’s well-documented and heavily represented in AI training data. I could literally describe what I wanted, and it would give me copy-paste-ready solutions. But with the P2P stack? Nope. Doesn’t work like that. These models weren’t trained enough on this ecosystem. They’ll hallucinate modules, give you outdated code, or just flat-out get things wrong. So I had to shift the way I code. Now, I focus on writing the **core functionality** myself. I don’t depend on LLMs to tell me how to use these modules — I do the research, read the source code if I have to, and write the initial logic myself. Once I’ve done that, and I’ve written enough code to give the model context, **then** I might use an LLM to help me scale something — like write repetitive patterns, generate variants of the same function, or help with edge cases. But I don’t let it suggest new modules or patterns unless it’s working strictly within the logic I already defined. Otherwise, it just derails everything. On top of that, I’ve started using a lot of dev tools I ignored before. Stuff like: - `budo` for live-reloading development servers - `browserify` to bundle code for the browser - `standardx` and `prettier` to keep code clean and consistent - `babel` for transpiling modern JS syntax - And tons of VS Code extensions that just make life easier Looking back, I realize how much time I wasted building full apps without any of these tools. Like, I would manually reload pages, deal with messy console logs, or just ignore code formatting — and that made debugging harder and slowed everything down. Now that I’ve integrated these tools, my workflow feels 10x faster and cleaner. </div> ----  <div style="font-size:20px;"> ### Final Thoughts So yeah, this stack is not for lazy devs or people who just want to copy code from StackOverflow and ship something. It forces you to actually **understand** what’s going on. But in a weird way, that’s what makes it fun. You get to build stuff at a lower level, understand networking concepts, and feel like you actually **earned** the result when something starts working. The learning curve is real, but the payoff is high — and I honestly don’t think I’ve ever learned this much from any other tech stack. It makes you more independent, more thoughtful, and less reliant on tools that only work when you’re following the mainstream. And yeah, still figuring it all out — but I’m enjoying the ride :) </div>