Basilisk UI Workshop - HackMD

<style> .present { text-align: left; } .reveal { font-size: 24px; } .reveal pre { width: 100%; } </style>  <center> # Basilisk UI Workshop ![](https://i.imgur.com/S6qtWPE.gif) <small>AmsterDOT 2022 | By Istvan Deak & Matej Sima</small> </center> --- ## What will we build / learn? 📚 -  Simplified DEX UI using Typescript, React and GraphQL. -  Basics of React (Functional components, Hooks, Context) -  UI Architecture & separation of concerns -  Basic understanding of Basilisk runtime source code (XYK palette) -  GraphQL + data composition from multiple sources -  Polkadot.js Typescript SDK - Reading of on-chain data - Submitting transactions - Executing custom RPC methods -  Polkadot.js Apps -  Updating data in 'real time' -  Building a Web3 backend with Subsquid -  Preparing historical data with an indexer/processor ---  <center> # Fireproof & inclusive <small>Anything to add? Just ask.</small> 🔥🚒👩‍🚒 </center> --- ## Audience scoping 🔭 -  Do you have any Javascript / Typescript experience? -  Do you have any experience with building UIs? If yes, what tools did you use? -  Do you have any experience with React? -  Do you have any experience with GraphQL? -  Do you have any experience with Rust or Substrate? -  Do you have any experience with Web3 apps, or DeFi (in the Dotsama landscape)? -  Do you wanna pair code, or just watch? <center> ![](https://i.imgur.com/8mHkNvF.gif)  </center> --- ## What is a DEX? 🤔 - Decentralized exchanges (DEX) allow users to exchange cryptocurrencies without giving up custody to a "trusted" intermediary - Basically users can trade assets from their crypto wallets directly - A DEX runs on a set of smart-contracts (eg. Uniswap), or the blockchain protocol itself (eg. Basilisk, HydraDX) - DEXs are either: - Order book-based - Liquidity pool-based (a.k.a. Automated Market Makers - **AMM**) - AMM DEXs allow users to trade assets 'on the spot' thanks to its algorithmic nature - We only focus on liquidity pool-based AMMs, because that is what we are building at Hydra <div class="fragment"> --- ## What is Basilisk? - DEX built as parachain within the Kusama ecosystem - The Basilisk 'protocol' a.k.a. runtime is composed from Paletts, which contain 'all' the necessary business logic. </div> --- ## What features should a simplified DEX UI include? 🤔 - Show a price for the given asset pair - Show a historical price for the given asset pair - Allow the user to buy or sell the given asset - Track the status of buy/sell transaction(s) ![](https://i.imgur.com/at9LKEz.png)  ---  <center> # #1: React bootcamp 🔫 ![](https://i.imgur.com/wAHAxfO.gif) </center> --- ## Setting up a React project 🧰 Prerequisites: - VSCode - Node.js v16 Bootstrap a react project: ```zsh # React v18 is really new, don't be scared $ npx create-react-app basilisk-ui-workshop --template typescript # Run it 🏃‍♀️ $ cd basilisk-ui-workshop $ npm start ``` --- ## Functional components 🔦 (1/2) What are functional components? - JS function that returns a JSX/TSX markup - Uses function arguments as 'properties' for the component - Runs each time the given component is rendered - Should be stateless / pristine - without any direct side-effects - Same input, should mean the same output ```tsx // src/components/Counter.tsx // Define the component first const Counter: React.FC<{}> = () => { return <p>Counter</p>; } ```  ```tsx // src/App.tsx const App: React.FC<{}> = () => { // render the component within 'App' return <Counter /> } ```  --- ## Functional components (2/2) 🎉 ![](https://i.imgur.com/Jd6MFNG.png) --- ## Managing component state (1/4) ```tsx export const Counter: React.FC<{}> = () => { // initialise the state as 0 const [count, setCount] = useState<number>(0); // return a <p> and render the state inside of it return <p>{count}</p> } ``` `useState` hook can help us persist state across component (re)renders. -  Generic, you can define a type for the stored state -  Can be initialised by providing an initial value -  Returns the persisted state and a setter for updating it ![](https://i.imgur.com/sfMaJON.png)  --- ## Managing component state (2/4) We want to provide a button to increment the counter: ```tsx export const Counter: React.FC<{}> = () => { const [count, setCount] = useState<number>(0); return <> <p>{count}</p> <button onClick={() => console.log('incrementing')}>Increment</button> </> } ``` `onClick` property of a JSX element: -  allows us to capture and handle user clicks - we can use `console.log` to verify the assumption above  - `⌘ ⌥ + J` to open the browser JS console --- ## Managing component state (3/4) Let's spice it up:  - Instead of just console-logging, we can increment our counter by calling the aforementioned state setter function  ```tsx export const Counter: React.FC<{}> = () => { const [count, setCount] = useState<number>(0); return <> <p>{count}</p> <button onClick={() => setCount(count => count + 1)}>Increment</button> </> } ```  - The snippet above has a problem, it redefines the `onClick` handler function each time the component (re)renders. We'll solve that next.  --- ## Managing component state (4/4) What can we improve? - `onClick` event handler is being redefined on every render, we know that's a problem  - `useCallback` can prevent re-creation of functions, as long as the function 'dependencies' do not change  ```tsx export const Counter: React.FC<{}> = () => { const [count, setCount] = useState<number>(0); const handleIncrementClick = useCallback(() => { setCount(count => count + 1) }, [setCount]); return <> <p>{count}</p> <button onClick={handleIncrementClick}>Increment</button> </> } ```  --- ## React component properties (1/3) Let's assume the following: - We want to re-use the `<Counter/>` component - We don't want it to increment by `1`, but by a different number at different occasions ```tsx const App: React.FC<{}> = () => { // Render two Counters, but wrap them in a fragment return <> <Counter /> <Counter /> </> } ```  What's new here? - React fragments `<></>` help us wrap multiple components into one parent without cluttering the DOM - We can see two Counters, each with their own separate state ✨ --- ## React component properties (2/3) ![](https://i.imgur.com/G1jiB1l.png) --- ## React component properties (3/3) Let's implement the following: - Add a property `incrementBy` to our `Counter`  - Render two `Counter(s)`, one with `incrementBy = 1` and one with `incrementBy = 2`  - We want to make sure the `incrementBy` property is optional  ```tsx // src/components/Counter.tsx // type of the Counter properties export interface CounterProps { incrementBy?: number // this will be number OR undefined, thanks to the `?` } ```  ```tsx // default `incrementBy = 1` export const Counter: React.FC<CounterProps> = ({ incrementBy = 1 }) => { const [count, setCount] = useState<number>(0); const handleIncrementClick = useCallback(() => { // increment using `incrementBy` setCount(count => count + incrementBy) }, [setCount]); return <> <p>{count}</p> <button onClick={handleIncrementClick}>Increment</button> </> } ```  --- ## React context & custom hooks (1/7) When multiple components depend on a single piece of state, you can do the following: - either pass it via props, and create a ton of code-coupling  - extract the given state using a React context, and subscribe to the context from within the component  Let's pretend that we want to implement the following: - A `magicalNumber` that will be added to the `count` each time we attempt to increment it  - The `magicalNumber` itself, will change every `x seconds` (e.g. 2 seconds)  --- ## React context & custom hooks (2/7) ![](https://i.imgur.com/QpK8Bq8.png) --- ## React context & custom hooks (3/7) Before we create the actual context, it'd be a good idea to figure out what value should it hold, and how the value can be changed. Our context will have the following features: - It will be a **custom hook**, promoted to a context - Generates a `magicalNumber` at the initial render as an initial value - Update the `magicalNumber` every `x seconds` - All the 'context subscribers' will be re-rendered each time the context value changes ```tsx // src/hooks/useMagicalNumber.tsx import { useCallback, useState } from "react" export const useMagicalNumber = () => { // generate a random number between 0-10 const generateMagicalNumber = useCallback(() => { return Math.round(Math.random() * 10); }, []); const [magicalNumber, setMagicalNumber] = useState<number>(() => generateMagicalNumber()); return magicalNumber; } ``` --- ## React context & custom hoooks (4/7) Assuming you want to try out the custom hook we've implemented previously: ```tsx /// src/App.tsx const App: React.FC<{}> = () => { const magicalNumber = useMagicalNumber(); return <> {magicalNumber} <Counter /> <Counter incrementBy={2} /> </> } ``` --- ## React context & custom hoooks (5/7) We've previously stated that we want the magicalNumber to change every x seconds, so let's do that: - We need to run a 'side effect' using a `useEffect` hook - This effect, will have start an interval using `setInterval`, that will generate and set the `magicalNumber` every x seconds ```tsx export const useMagicalNumber = () => { // generate a random number between 0-10 const generateMagicalNumber = useCallback(() => { return Math.round(Math.random() * 10); }, []); // initialize the magicalNumber state const [magicalNumber, setMagicalNumber] = useState<number>(() => generateMagicalNumber()); // run a side effect, but just once useEffect(() => { // run a function every 1 second setInterval(() => { // re-generate a new `magicalNumber` const magicalNumber = generateMagicalNumber(); // update the state setMagicalNumber(magicalNumber); }, 1000); }, [setMagicalNumber, generateMagicalNumber]); return magicalNumber; } ``` --- ## React context & custom hoooks (6/7) As a next step, we want to: - Promote the custom hook to a context, this will include creating a Context Provider and a hook to consume the context value - Update the `Counter` component to consume the `magicalNumber` context First we'll install `constate`, a little library that turns hooks into contexts: ```shell-session npm i constate --legacy-peer-deps ``` ```tsx // src/hooks/useMagicalNumber.tsx export const [MagicalNumberProvider, useMagicalNumberContext] = constate<PropsWithChildren<any>, ReturnType<typeof useMagicalNumber>, never>(useMagicalNumber) ``` ```tsx // src/components/Counter.tsx export const Counter: React.FC<CounterProps> = ({ incrementBy = 1 }) => { const magicalNumber = useMagicalNumberContext(); const [count, setCount] = useState<number>(0); const handleIncrementClick = useCallback(() => { setCount(count => count + incrementBy + magicalNumber); }, [setCount, magicalNumber]); return <> <p>{count}</p> <button onClick={handleIncrementClick}>Increment</button> </> } ``` --- ## React context & custom hoooks (7/7) Before you run the code from the previous slide, you must update the `App` component to use the `MagicalNumberProvider`. ```tsx // src/App.tsx const App: React.FC<{}> = () => { return <> <MagicalNumberProvider> <Counter /> <Counter incrementBy={2} /> </MagicalNumberProvider> </> } ``` --- ## Counter App Architectural overview ![](https://i.imgur.com/k6r8U1Q.png) --- ## React bootcamp recap What did we learn so far? - Install NodeJS - Setup a React project using `create-react-app` - Install dependencies - Run the development server - Basics of React functional components - Component definition, returning a markup to be rendered, fragments - Basics of React Hooks, namely: - `useState` - `useCallback` - `useEffect` - ... more examples would be e.g. `useMemo` - Basics of React context & constate - Handling of user events / interaction with our components ---  <center> # #2 Chamber of secrets 🐍 ![](https://i.imgur.com/Bn2EFLi.gif) </center> --- ## What is a *Runtime*? In simple terms, the runtime itself is: - It's smart contracts on steroids  - A collection of Rust code, known as palettes (and more) - Palettes generally hold business logic and define:  - How data is stored (chain state)  - How the business logic is interacted with (extrinsics)  - How 3rd party can *observe* stuff happening on the chain (events)  - There's more to it, but you better listen to an actual runtime engineer than me 🔥🧯  --- ## Where does Basilisk come in? - Basilisk runs as a parachain connected to Kusama - For the workshop purposes, we have our own instance of Basilisk instance running (+ a publicly available testnet instance) - We will use Basilisk to: - Create an AMM/DEX pool to trade in - Read interesting pool data from the Basilisk node - Submit trades (buy) to the Basilisk node, which will be executed in our pool --- ## Connecting to the Testnet / Workshopnet <center> ![](https://i.imgur.com/KW5IFw0.gif) </center> --- ## Connecting to the Testnet / Workshopnet - Don't worry about running the whole infrastructure stack on your computer - We are providing a private Basilisk instance for this workshop - chain: `wss://amsterdot.eu.ngrok.io` - archive: `https://amsterdot-archive.eu.ngrok.io` - processor: `https://amsterdot-processor.eu.ngrok.io` - This instance is not the public testnet - We can easily reset the whole chain throughout the workshop - We *migrate* a state consisting of 2 XYK pools --- ## Overview of the Basilisk-node palettes ![](https://i.imgur.com/M1RU461.png) --- ## Basilisk XYK pools (1/2) We know what is an AMM, but what are XYK pools? - From an architectural perspective, pool is an entity that serves as a gateway for a trade  - It **contains 2 assets**, e.g. BSX and DAI, which can be traded within the pool  - The pool itself holds a balance of both of its assets  - **Relationship between the two asset balances is known as the spot price**  - When an asset comes out (buy), another asset needs to 'go in'.. and vice versa (sell).  - Trades 'happen' as soon as the transaction is included in the block and is successful.  --- ## Basilisk XYK pools (2/2) ![](https://i.imgur.com/Scjjwua.png) --- ## Polkadot.js Apps (1/5) One of the most comprehensive ways to interact with a substrate based node, is using the Polkadot.js apps interface. It allows us to: - Browse chain state per palette - Interact with the RPC - Submit extrinsics (transactions / palette function calls) - ... and more - Verify our code/values with a 3rd party app ---  <center> ### Pull out your magic wand and let's get practical 🪄 [Open Polkadot.js apps](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Fbasilisk-rpc.hydration.cloud%2F#/chainstate) </center> Note: Walk through reading pools from the testnet (XYK), how balances can be read. We will show submitting a trade later. Mention that we will come back to Polkadot.js apps pretty often to double check stuff. --- ## Polkadot.js Apps (2/5) What data can we easily take a look at using Polkadot.js Apps, and what are the caveats? Let's take XYK pools as an example: ```code xyk.poolAssets(AccountId32): Option<(u32, u32)> ```  - `xyk` - palette name  - `poolAssets` - storage name  - `AccountId32` - type of the key in the storage 'map'.. it's just an address  - `Option<(u32, u32)>` - type of the value in the storage map  - `Option` means that the value may or may not be there, so it could be of type `Some(value) / None`  - `u32` is an integer (unsigned? IDK, ask a rust dev)  Feeling adventurous? Check out the Rust implementation directly [here](https://github.com/galacticcouncil/Basilisk-node/blob/master/pallets/xyk/src/lib.rs#L247).  --- ## Polkadot.js Apps (3/5) ```json [ [ // pool address [ bXkC7B8CZ4256NwNRbRcEJhxSZSxUtDPzm6cALj9xTDdihDRj ] // pool asset IDs [ 0 3 ] ] [ [ bXjT2D2cuxUuP2JzddMxYusg4cKo3wENje5Xdk3jbNwtRvStq ] [ 0 2 ] ] [ [ bXn6KCrv8k2JV7B2c5jzLttBDqL4BurPCTcLa3NQk5SWDVXCJ ] [ 0 1 ] ] ] ``` --- ## Polkadot.js Apps (4/5) We can fetch balances for a pool in two ways, depending on what assets does the pool contain. Native asset (0): ```code system.account(AccountId32): FrameSystemAccountInfo ``` ```json { nonce: 0 consumers: 0 providers: 2 sufficients: 0 data: { free: 10,000,926,848,938,544,136,618 reserved: 0 miscFrozen: 0 feeFrozen: 0 } } ``` But... that's a lot of tokens isn't it? Yes, they are stored with a `10^12` precision, so to get an actual number of tokens you have to do the following: ```code 10,000,926,848,938,544,136,618 / 10^12 = still a lot ``` --- ## Polkadot.js Apps (5/5) 3rd party assets (1+): ```code tokens.accounts(AccountId32, u32): OrmlTokensAccountData ``` ```json [ [ [ bXn6KCrv8k2JV7B2c5jzLttBDqL4BurPCTcLa3NQk5SWDVXCJ 1 ] { free: 999,907,706,920,665,435,037 reserved: 0 frozen: 0 } ] ] ``` ---  <center> ## But.. how do we calculate the spot price? ![](https://i.imgur.com/NuX5hgA.gif) </center> --- ## Calculating the spot price So far we've learned what pools are and how do they fit into the runtime data model. Assuming the following: - We have a BSX/DAI pool - It holds `1000 BSX` (for the sake of easy calculation) - And `2000 DAI` Then we the spot price can be calculated as follows: ```code 1 BSX = 2000 DAI / 1000 BSX = 2 DAI 1 DAI = 1000 BSX / 2000 DAI = 0.5 BSX OR 1 A = LIQUIDITY_B / LIQUIDITY_A ``` *The actual trade price will differ by the price impact, depending on the depth of liquidity. Take a look at [the HydraDX math library](https://github.com/galacticcouncil/HydraDX-math/blob/main/src/xyk.rs#L46) to learn more.* ---  <center> # #3 UI Architecture & Implementation 🏰♀️ ![](https://i.imgur.com/AHuc4do.gif) </center> --- ## Separation of concerns How can we effectively split an app, to distinguish and distribute responsibilities between certain pieces of code? - Data layer  - Compositional layer  - Presentational layer  Pros ✅  - Easy debugging  - Seperation of resposibility = separation of work (between contributors)  - Testing  - Codebase clarity, easier to reason about implementation details  Cons ❌  - Harder to see the benefits in the short term  - Possibly more boilerplate code (oh no)  - Operational overhead (more work manage, additional PRs, tickets and such)  ---  <center> # Data layer </center> --- ## Introducing GraphQL What is GraphQL? - A query language (data go brrr 🚂)  - Besides that also a set of tools to build data-first clients  - Or.. even servers to support those clients  - It can also generate client-side schemas, so you know what you're working with  - Originally built by Facebook yaaay! (or? ➡ It's MIT licensed, don't worry)  --- ## Getting started with Apollo Client Since we're building a React based app, we need a GraphQL library with React support, Apollo Client is one of those. What does it offer us? - A layer of abstraction and unification for working with remote data sources - A **normalized cache solution** - **Local state management** - Basic type/code generator from a GraphQL schema You can install it with: ```zsh $ npm i @apollo/client graphql @types/graphql --legacy-peer-deps ``` You can install Apollo Client Devtools from [here](https://chrome.google.com/webstore/detail/apollo-client-devtools/jdkknkkbebbapilgoeccciglkfbmbnfm). --- ## Architectural overview with Apollo Client ![](https://i.imgur.com/UkVSw8w.png) --- ## Setting up our data model In order to get going with our data layer, we need to define a data model for our UI first. ```typescript // src/lib/pool.tsx export type Address = string; export type AssetId = string; // tuple of two asset IDs export type AssetPair = [AssetId, AssetId]; export interface Pool { // entity identifier for Apollo cache __typename: string, // pool address id: Address, // ordered tuple of asset IDs assets: AssetPair, } ``` --- ## Writing our first query (1/2) What is a GraphQL resolver, and how does apollo local state management work? - GraphQL resolver is a function that *resolves* a piece of GraphQL query, more specifically a single entity or a field - We want to be able to fetch a specific/single pool - while the only information we have availabele on the UI is the two asset IDs that the pool might hold - We need to use the `@client` annotation to let Apollo know we want to resolve the given entity/field locally (on the client side) ```graphql= # Query that looks for a pool by a tuple of assets # Note: GraphQL doesnt have tuples, just arrays, # but country girls make do 🌽 query GetPoolByAssets($assets: [String!]!) { # ⚠️ pass the parameter to the resolver here pool(assets: $assets) @client { id, assets, } } ``` ```typescript= // src/hooks/useGetPoolByAssetsQuery.tsx export const getPoolByAssetsQuery = gql` <query from above goes here> ` ``` --- ## Writing our first query (2/2) The query we've written above needs to be exposed via a hook, otherwise we could hardly use it inside a container. We need to: - Define what data is returned by the query - What parameters does the query accept - Expose a hook, using the above type definitions ```typescript // src/hooks/useGetPoolByAssetsQuery.tsx // we need to know what kind of data does the query return export interface GetPoolByAssetsQueryResponse { // remember what function we're using to extract the type here, // we'll come back to this later pool: Awaited<ReturnType<typeof getPool>> } // what kind of variables does the query require export interface GetPoolByAssetsQueryVariables { assets: AssetPair } // useQuery only works inside of an ApolloProvider context export const useGetPoolByAssetsQuery = (variables: GetPoolByAssetsQueryVariables) => useQuery<GetPoolByAssetsQueryResponse, GetPoolByAssetsQueryVariables>( getPoolByAssetsQuery, { variables } ); ``` --- ## Time to consume our data Let's do the following: - Define a new container called `Trade` - Consume the previously defined query - Keep track of the current `AssetPair` ```typescript= // src/containers/Trade.tsx export const defaultAssetPair: AssetPair = ['0', '1']; export const Trade: React.FC<{}> = () => { const [assets, setAssets] = useState<AssetPair>(defaultAssetPair); const { data: poolData, loading, error } = useGetPoolByAssetsQuery({ variables: { assets } }); // let's console log a bunch of stuff for debugging console.log('Trade', { assets, poolData, loading }) // useful for debugging the resolver error && console.error(error) return <></> } ``` ```typescript= const App: React.FC<{}> = () => { return <> <Trade /> </> } ``` > Oops, this won't work... maybe in three weeks™️? --- ## Setting up the Apollo Client Why doesn't it work??? - `useQuery` must be wrapped in an `ApolloProvider` in order to work - We have to define a resolver to actually *resolve* the data we're asking for - Set up an `ApolloClient` instance, and pass it to the `ApolloProvider` ```typescript= // src/hooks/useApolloClient.tsx export const useApolloClient = (): ApolloClient<NormalizedCacheObject> => { return useMemo(() => { return new ApolloClient({ uri: '', // if you're consuming an external GraphQL resource, use this cache: new InMemoryCache() }) }, []); } ``` ```typescript= // src/App.tsx const App: React.FC<{}> = () => { const client = useApolloClient(); return <> <ApolloProvider client={client}> <Trade /> </ApolloProvider> </> } ``` --- ## Let's resolve the query (1/2) As you may have noticed, the query currently does not return any data, let's do the following: - Implement a function `getPool` that returns a mock pool - Register `getPool` as a resolver for the `pool` entity/field ```typescript= // src/lib/pool.tsx export const __typename = 'Pool'; export const getPool: Resolver = async (): Promise<Pool | undefined> => { return { __typename, id: '1', assets: ['0', '1'], } } ``` ```typescript= // src/hooks/useApolloClient.tsx return new ApolloClient({ uri: '', cache: new InMemoryCache(), resolvers: { // resolve a `pool` within a `Query` using `getPool` Query: { pool: getPool } } }) ``` --- ## Let's resolve the query (2/2) Did you notice anything odd? - The query executes twice, why is that? - Our app is running in `React.StrictMode`, which makes it render twice. - `useGetPoolByAssetsQuery` is also called twice - Due to the functional nature of our codebase, we'll have to keep an eye out for *double-dipping* - This might be very helpful in certain cases, but we'll disable it to avoid double-running our queries and resolvers. ```typescript= // src/index.tsx // get rid of <React.StrictMode> root.render( <App /> ); ``` That's it, it no longer runs twice 🎉 --- ## Resolving the requested data (1/4) We know that our frontend will need to query a `Pool` by an `AssetPair`. It's how we've structured our query previously. However we can't really *lookup* a pool by it's assets directly - we'll need to find/compute it's ID first. This can be done with the help of RPC, but let's setup the resolver structure first. We'll do the following: - get rid of `getPool` and introduce `getPoolById` and `getPoolId` - ⚠️ Don't forget to update your query type definition to use `getPoolById`, instead of `getPool` - introduce a `poolResolver`, that translates the `assets` parameter into an ID and then fetches a pool --- ## Resolving the requested data (2/4) ```typescript // src/lib/pool.tsx export const getPoolById = async (id: string): Promise<Pool | undefined> => { return { __typename, id: '1', assets: ['0', '1'], } } export const getPoolId = async (assets: AssetPair): Promise<string | undefined> => { return 'mock-id'; //for now } ``` ```typescript= export interface PoolResolverArgs { assets: AssetPair } export const poolResolver: Resolver = async (_entity: unknown, { assets }: PoolResolverArgs, context: unknown): Promise<Pool | undefined> => { // get the ID of the pool by its assets const id = await getPoolId(assets); // if we can't find the requested pool, throw an error if (!id) throw new Error('Pool not found'); // proceed with fetching the pool return getPoolById(id); } ``` --- ## Resolving the requested data (3/4) We've already built the resolver, which wraps our internal data library. It's time to integrate it into our Apollo Client: - Create a new `useResolvers` hook - Set resolvers for the `client`, also if they change (thanks to `useEffect`) - This is a stepping stone for dependency injection and resolver memoization 🪜 - Since our future dependencies will be *contextual* a.k.a. depending on the react context, we need our client & resolver setup to have access to the context as well. - Keeping our data layer within the confines of React gives us additional control over e.g. disconnecting from (websocket) data sources once we don't need them anymore (e.g. when a component is dismounted) > Rule of thumb is keep the `lib` as clean as possible, and only introduce hooks when you actually need them (e.g. to access contextual dependencies) --- ## Resolving the requested data (4/4) ```typescript // src/hooks/useApolloClient.tsx export const useResolvers = (): Resolvers => { return useMemo(() => { return { Query: { pool: poolResolver } } }, []) // notice the empty dependency array } ``` ```typescript // src/hooks/useApolloClient.tsx export const useApolloClient = (): ApolloClient<NormalizedCacheObject> => { const client = useMemo(() => { return new ApolloClient({ uri: '', cache: new InMemoryCache(), // you need at least an empty object here, // otherwise @client data aren't resolved resolvers: {} }) }, []); const resolvers = useResolvers(); // set resolvers initially and when they change useEffect(() => { client.setResolvers(resolvers); }, [client, resolvers]) return client } ``` ---  <center> # Dependencies ![](https://i.imgur.com/xe7sqL1.gif) </center> --- ## Goodbye mock data, welcome Polkadot.js SDK (1/3) So far we've just consumed a bunch of mock data through our queries and resolvers. We only needed a place holder to connect a few dots together. In reality you'd want to fetch the given pool in the same way as you did with Polkadot.js apps. How can we conceptually fit this into our stack? - We'll use the Polkadot.js SDK (a.k.a. API) - We'll introduce dependency injection in our resolvers - We'll create a new `DependencyProvider` along with a context, where we'll inject an instance of the Polkadot.js SDK - ❓ We'll also install `@open-web3/orml-types`, since it contains many useful type definitions that are used in the Basilisk runtime You can install the Polkadot.js SDK like this: ```shell-session npm i @polkadot/api @open-web3/orml-types ``` ```typescript= // src/hooks/useDependencies.tsx // shape of our dependency state export interface Dependencies { polkadotJs?: ApiPromise } export const useDependencies = (): Dependencies | undefined => { // keep the existing dependency instances in our state const [dependencies, setDependencies] = useState<Dependencies>(); return dependencies; } export const [DependencyProvider, useDependencyContext] = constate<PropsWithChildren<any>, ReturnType<typeof useDependencies>, never>(useDependencies) ``` --- ## Goodbye mock data, welcome Polkadot.js SDK (2/3) In order to use the Polkadot.js SDK, we first need to instantiate it. ```typescript= // src/lib/polkadotJs.tsx export const createInstance = async (): Promise<ApiPromise> => { // connect to our workshop network // public testnet node: 'wss://basilisk-rpc.hydration.cloud/' // workshop node: 'wss://amsterdot.eu.ngrok.io' const provider = new WsProvider('wss://basilisk-rpc.hydration.cloud/') // create the SDK instance const instance = await ApiPromise.create({ provider }); // wait for the instance to finish setting up (e.g. connecting) await instance.isReady; return instance; } ``` --- ## Goodbye mock data, welcome Polkadot.js SDK (3/3) Next step would be to include the Polkadot.js SDK as part of our dependencies: - We'll create a hook called `useCreatePolkadotJs` - this hook will instantiate the SDK, and once it's ready it'll be set as a dependency ```typescript= // src/hooks/useDependencies.tsx export const useCreatePolkadotJs = (setDependencies: Dispatch<SetStateAction<Dependencies | undefined>>) => { useEffect(() => { (async () => { const polkadotJs = await createInstance(); setDependencies(dependencies => { return ({ ...dependencies, polkadotJs }) }) })() }, [setDependencies]) }; ``` --- ## Consuming dependencies (1/4) We've managed to bootstrap our dependencies, now it's time to put them to work. - Setup our resolvers, therefore the Apollo Client to have access to contextual dependencies - Upgrade from `ApolloProvider` to `ConfiguredApolloProvider` - Any component *consuming* a context needs to be nested in the given context's provider - We want `useApolloClient` to be able to access our dependencies with `useDependencyContext` ```typescript= // src/hooks/useApolloClient.tsx export const ConfiguredApolloProviderReact.FC<PropsWithChildren<{}>> = ({ children }) => { const client = useApolloClient(); return <ApolloProvider client={client}> {children} </ApolloProvider> } ``` ```typescript= // src/App.tsx const App: React.FC<{}> = () => { return <> <DependencyProvider> <ConfiguredApolloProvider> <Trade /> </ConfiguredApolloProvider> </DependencyProvider> </> } ``` --- ## Consuming dependencies (2/4) ```typescript= // src/hooks/useApolloClient.tsx export const useResolvers = (dependencies?: Dependencies): Resolvers => { return useMemo(() => { return { Query: { pool: poolResolver } } }, []) } export const useApolloClient = (): ApolloClient<NormalizedCacheObject> => { const client = useMemo(() => { return new ApolloClient({ uri: '', cache: new InMemoryCache(), // you need at least an empty object here, // otherwise @client data aren't resolved resolvers: {} }) }, []); // pass dependencies to resolvers const dependencies = useDependencyContext(); const resolvers = useResolvers(dependencies); // set resolvers initially and when they change useEffect(() => { client.setResolvers(resolvers); }, [client, resolvers]) return client } ``` --- ## Consuming dependencies (3/4) ```typescript= // src/hooks/useApolloClient.tsx export const useResolvers = (dependencies?: Dependencies): Resolvers => { return useMemo(() => { return { Query: { pool: poolResolver(dependencies?.polkadotJs) } } }, [dependencies]) } ``` ```typescript= // src/lib/pool.tsx export const poolResolver = (apiInstance?: ApiPromise): Resolver => async (_entity: unknown, { assets }: PoolResolverArgs, context: unknown): Promise<Pool | undefined> => { // throw an error if the dependency isn't ready yet if (!apiInstance) throw new Error('SDK instance is not ready yet') // get the ID of the pool by its assets const id = await getPoolId(assets); // if we can't find the requested pool, throw an error if (!id) throw new Error('Pool not found'); // proceed with fetching the pool return getPoolById(id); } ``` --- ## Consuming dependencies (4/4) You might have noticed that the query now errors out, since the dependencies for the resolver are not ready yet. We can solve that by introducing a simple hook called `useDependenciesLoading` that keeps track of all outstanding dependencies, and postpones the query until they are ready. ⚠️ You might be tempted to just use `useMemo`, but this might result into a wierd race condition where the `Trade` component will think the dependencies are not loading anymore, but the resolvers won't be ready yet. ```typescript= // src/hooks/useDependencies.tsx export const useDependenciesLoading = (): boolean => { const dependencies = useDependencyContext(); const [loading, setLoading] = useState(true); useEffect(() => { if (!dependencies?.polkadotJs) return; setLoading(false); }, [dependencies, setLoading]); return loading; } ``` ```typescript= // src/containers/Trade.tsx ... const dependenciesLoading = useDependenciesLoading(); const { data: poolData, loading: poolDataLoading, error } = useGetPoolByAssetsQuery({ variables: { assets }, skip: dependenciesLoading }); const loading = useMemo(() => dependenciesLoading || poolDataLoading, [dependenciesLoading, poolDataLoading]); ``` ---  <center> # Fetching real data ![](https://i.imgur.com/1TobGR1.gif) </center> --- ## Using the Polkadot.js SDK (1/3) We already have a `Trade` container that runs a query fetching a `Pool` by `assets`. What we want to do next, is serve real on-chain data instead of mocks: - Setup our SDK to work with a custom RPC method to determine the pool - Pool id is calculated from the `poolAssets`, e.g. 0/1 would make up a pool id of `bXn6KC...SWDVXCJ` - Use the SDK to fetch the pool by ID from the XYK palette ```typescript= // src/lib/polkadotJs.tsx const rpc: ApiOptions['rpc'] = { xyk: { getPoolAccount: { description: 'Get pool account id by asset IDs', params: [ { name: 'assetInId', type: 'u32', }, { name: 'assetOutId', type: 'u32', }, ], type: 'AccountId', }, }, }; export const createInstance = async (): Promise<any> => { const provider = new WsProvider('wss://amsterdot.eu.ngrok.io') const instance = await ApiPromise.create({ provider, rpc }); // wait for the instance to finish setting up (e.g. connecting) await instance.isReady; return instance; } ``` --- ## Using the Polkadot.js SDK (2/3) In order to determine the pool id we need to: - Wrap polkadot.js with a custom type definition, including our custom RPC method - Call our RPC method and use the resulting ID to fetch the pool - ⚠️ Be careful, the RPC method *only* calculates the pool id, even if the pool does not exist. - ⚠️ Don't forget to update your `poolResolver` to pass along the `apiInstance` to `getPoolId` (and `getPoolbyId`) ```typescript= // src/lib/pool.tsx // we don't know the return type (CustomRPC) up front :/ export const customRPC = (apiInstance: ApiPromise) => { type CustomRPC = { xyk: { getPoolAccount: (assetOutId: string, assetInId: string) => AccountId; } } & typeof apiInstance.rpc; return apiInstance.rpc as CustomRPC; } export const getPoolId = (apiInstance: ApiPromise) => async (assets: AssetPair): Promise<Pool['id'] | undefined> => { const poolId = await customRPC(apiInstance).xyk.getPoolAccount(assets[0], assets[1]); return poolId.toString(); } ``` --- ## Using the Polkadot.js SDK (3/3) Finally, we have the pool id - which means we can fetch our pool: - Query for `xyk.poolAssets` using the pool id obtained earlier - Combine the id & pool assets to acquire the desired `Pool` type we want to return from our resolver. - Define a custom `PoolAssets` type, to describe the storage we're querying - Since we're using an `Option`, we make sure it's `Some` ```typescript= // src/lib/pool.tsx // is there a better way to define those types? 🤔 export type PoolAssets = IOption<ITuple<[U32, U32]>>; // transform id & poolAssets into a Pool type export const toPool = (id: Pool['id'], poolAssets: PoolAssets): Pool | undefined => { if (poolAssets.isSome) { return { id, __typename, assets: [ poolAssets.unwrap()[0].toString(), poolAssets.unwrap()[1].toString() ] } } } // fetch poolAssets by an id export const getPoolById = (apiPromise: ApiPromise) => async (id: Pool['id']): Promise<Pool | undefined> => { const poolAssets = await apiPromise.query.xyk.poolAssets<PoolAssets>(id); return toPool(id, poolAssets); } ``` --- ## Apollo dev tools Assuming your query is working and the data is shown, you should be able to see it via the Apollo Dev Tools as well. ![](https://i.imgur.com/3ZdbOFY.png) ---  <center> # Presentational layer <small>Without CSS ✨</small> ![](https://i.imgur.com/OEBq4Hd.gif) </center> --- ## Displaying our pool (1/3) Let's build a small presentational component, that will take care of displaying our pool data. ```typescript= // src/components/Pool.tsx export interface PoolProps { pool?: PoolModel, loading: boolean } export const Pool: React.FC<PoolProps> = ({ pool, loading }) => { return <> {!loading ? pool ? <> <p>ID: {pool.id}</p> <p>Assets: {pool.assets[0]} / {pool.assets[1]}</p> <p>Spot price: ???</p> </> : 'No pool found :/' : 'loading...' } </> } ``` ```typescript= // src/containers/Trade.tsx ... return <> <h1>Basilisk UI Workshop</h1> <Pool pool={poolData?.pool} loading={loading} /> </> ``` --- ## Displaying our pool (2/3) ![](https://i.imgur.com/BlfHacA.png) --- ## Displaying our pool (3/3) ![](https://i.imgur.com/ubS7WME.png) --- ## Changing the AssetPair (1/7) Let's build a form that will let us change the `AssetPair` stored in the `Trade` component: - Using `react-hook-form - Standalone `Form` component, that displays the `assets` from the `Trade` component - Can bubble up a change of the `assets` back to the `Trade` component, so it can re-run our query with new variables ```typescript= // src/components/Form.tsx export interface FormProps { assets: AssetPair | undefined, onAssetsChange: (assets: AssetPair) => void } export const Form: React.FC<FormProps> = ({ assets }) => { return <></> } ``` --- ## Changing the AssetPair (2/7) Update our `Trade` container to render the `Form` as well. ```typescript= // src/containers/Trade.tsx ... return <> <h1>Basilisk UI Workshop</h1> <Pool pool={poolData?.pool} loading={loading} /> <Form assets={assets} onAssetsChange={(assets) => console.log('onAssetsChange', assets)}/> </> ``` --- ## Changing the AssetPair (3/7) As we've previously learned, `useCallback` is the way to go when dealing with callbacks. ```typescript= const handleAssetsChange = useCallback((assets: AssetPair) => { console.log('handleAssetsChange', assets); }, []) return <> <h1>Basilisk UI Workshop</h1> <Pool pool={poolData?.pool} loading={loading} /> <Form assets={assets} onAssetsChange={handleAssetsChange}/> </> ``` One last change we'll add is actually calling `setAssets` when they change within the form. ```typescript= const handleAssetsChange = useCallback((assets: AssetPair) => { setAssets(assets); }, []) ``` --- ## Changing the AssetPair (4/7) In order to actually change the `assets`, we'll need a form. Let's install `react-hook-form` to make things a lot easier. ```shell-session npm i react-hook-form ``` We'll start by defining the structure of our form: ```typescript= // src/components/Form.tsx export interface FormFields { assetIn: string assetOut: string } export const Form = ({ assets }: FormProps) => { const form = useForm<FormFields>({ defaultValues: assets && { assetIn: assets[0], assetOut: assets[1] } }); ... } ``` --- ## Changing the AssetPair (5/7) Let's render a two inputs for our assetIds, and a submit button. We'll use `react-hook-form` to handle the form submission too. ```typescript= const handleSubmit = useCallback((data: FormFields) => { console.log('data', data) }, []); return <div> <form onSubmit={form.handleSubmit(handleSubmit)}> <input type="text" {...form.register('assetIn')} /> <input type="text" {...form.register('assetOut')} /> <input type="submit" /> </form> </div> ``` --- ## Changing the AssetPair (6/7) We'd like to actually propage the form values back to `Trade`, so we can fetch the pool by the new `assets`. Let's setup some basic form validation first, in order to prevent fetching the pool for invalid `assets`. Let's change the validation configuration to run on every field change first: ```typescript= // src/components/Form.tsx const form = useForm<FormFields>({ defaultValues: assets && { assetIn: assets[0], assetOut: assets[1] }, mode: 'onChange' }); ... ``` Mark both form fields as required, and display the validation message: ```typescript= // src/components/Form.tsx ... return <div> <form onSubmit={form.handleSubmit(handleSubmit)}> <input type="text" {...form.register('assetIn', { required: true })} /> <input type="text" {...form.register('assetOut', { required: true })} /> <p> {!form.formState.isValid ? 'oops, you need to input both assetIds' : <></> } </p> <input type="submit" /> </form> </div> ``` --- ## Changing the AssetPair (7/7) Finally, once the form is submitted we can pass the data along to the parent component: ```typescript= // src/components/Form.tsx const handleSubmit = useCallback((data: FormFields) => { onAssetsChange([data.assetIn, data.assetOut]) }, [onAssetsChange]); ... ``` ![](https://i.imgur.com/Xg9oVU2.png) ---  <center> # Give us the spot price already ![](https://i.imgur.com/BlD7B9z.gif) </center> --- ## Fetching balances for an address (1/7) In order to calculate the spot price, we need to fetch balances for both assets in the pool. This means we're going back to the data layer. We've learned the differences between native and 3rd party assets earlier, this means we have to do two Polkadot.js SDK calls a.k.a. chain state queries. We'll register a new resolver for `balances`, and it will act as a sub-field or a nested resolver. This means it requires a parent entity which we'll use to extract the address to fetch balances for. This is how our updated pool query will eventually look like: ```graphql= query GetPoolByAssets($assets: [String!]!) { # ⚠️ pass the parameter to the resolver here pool(assets: $assets) @client{ id, assets, # if you're feeling playful, check out Apollo's '@export(as: "...") balances(assets: $assets) { assetId, free } } } ``` --- ## Fetching balances for an address (2/7) We need two lib functions, one to fetch the native asset balance. And one to fetch the remaining token balances. For the token balances, we'll also need to specify what assetIds we want to fetch. ```typescript= // src/lib/balances.tsx // generic Balance for both native and 3rd party assets export type Balance = { __typename: string, id: string, assetId: AssetId, free: string } export const __typename = 'Balance'; export const getNativeBalance = (apiInstance: ApiPromise) => async (address: string): Promise<Balance> => { const accountInfo = await apiInstance.query.system.account(address); return { __typename, id: `0-${address}`, assetId: '0', free: accountInfo.data.free.toString() } } ``` --- ## Fetching balances for an address (3/7) ```typescript= export const getTokensBalances = (apiInstance: ApiPromise) => async (address: string, assetIds: AssetId[]): Promise<Balance[]> => { // get rid of the native asset, since it's representation in the `tokens` palette isn't correct assetIds = assetIds.filter(assetId => assetId !== '0'); const queryParameter: string[][] = assetIds.map((assetId) => [ address, assetId, ]); const accountData = await apiInstance.query.tokens.accounts.multi<OrmlAccountData>( queryParameter ); return accountData.map((accountData, i) => { const assetId = assetIds[i]; return { __typename, id: `${assetId}-${address}`, assetId, free: accountData.free.toString() } }) } ``` --- ## Fetching balances for an address (4/7) As the next step we'll define a `balancesResolver` which will return a normalised `Balance` for both native and 3rd party assets/tokens. ```typescript= // src/lib/balances.tsx export interface PoolResolverArgs { assets: AssetId[] } export const balancesResolver = (apiInstance?: ApiPromise): Resolver => async (entity: ParentEntity, { assets }: PoolResolverArgs): Promise<Balance[] | undefined> => { if (!apiInstance) throw new Error('SDK instance is not ready yet'); const nativeBalance = await getNativeBalance(apiInstance)(entity.id); const tokenBalances = await getTokensBalances(apiInstance)(entity.id, assets) return [nativeBalance].concat(tokenBalances); } ``` --- ## Fetching balances for an address (5/7) TODO: move this slide to be one of the first ones, so you can actually run the resolver code as you go Finally, we can register our resolver and update our query. We'll register `balances` as a sub-field of `Pool`. ```typescript= // src/hooks/useApolloClient.tsx export const useResolvers = (dependencies?: Dependencies): Resolvers => { return useMemo(() => { return { Query: { pool: poolResolver(dependencies?.polkadotJs) }, Pool: { balances: balancesResolver(dependencies?.polkadotJs) } } // todo: leave this empty and showcase how hook dependencies need to be defined bcs bugs }, [dependencies]) } ``` ```typescript= // src/hooks/useGetPoolByAssetsQuery.tsx export const getPoolByAssetsQuery = gql` # Query that looks for a pool by a tuple of assets # Note: GraphQL doesnt have tuples, just arrays, # but country girls make do 🌽 query GetPoolByAssets($assets: [String!]!) { # ⚠️ pass the parameter to the resolver here pool(assets: $assets) @client{ id, assets, balances(assets: $assets) { assetId, free } } } ` ``` --- ## Fetching balances for an address (6/7) Last but not least, let's show the balances to see if everything is working as expected. We'll also have to update the params of our `Pool` component, since we're now also showing balances. ```typescript= // src/components/Pool.tsx export interface PoolProps { pool?: GetPoolByAssetsQueryResponse['pool'], loading: boolean } export const Pool = ({ pool, loading }: PoolProps) => { return <> {!loading ? pool ? <> <p>ID: {pool.id}</p> <p>Assets: {pool.assets[0]} / {pool.assets[1]}</p> <div> <p>Balances:</p> {pool.balances.map(balance => ( <div key={balance.assetId}> <p>AssetId: {balance.assetId}</p> <p>Free balance: {balance.free}</p> </div> ))} </div> <p>Spot price: ???</p> </> : 'No pool found :/' : 'loading...' } </> } ``` --- ## Fetching balances for an address (7/7) ![](https://i.imgur.com/mqM6dNp.png) ---  <center> # Field policies ![](https://i.imgur.com/0rcF8yx.gif) </center> --- ## Spot price as a field policy (1/6) Field policies allow us to calculate data on the fly using e.g. data from the cache. If you've worked with redux, it's very similiar to selectors. We'll use field policies to calculate the spot price from the balances we've fetched for our pool. But first let's implement pool spot price calculation. Start by installing `bignumber.js`: ```shell-session npm i bignumber.js ``` ```typescript= // src/lib/pool.tsx // given a pool (just balances), calculate the spot price // with the given order of assets (one way or another) export const calcualateSpotPrice = (pool: Balances, assets: AssetPair) => { const assetA = assets[0]; const assetB = assets[1]; const assetABalance = pool?.balances.filter(balance => balance.assetId === assetA)[0].free const assetBBalance = pool?.balances.filter(balance => balance.assetId === assetB)[0].free // This can go really bad - e.g. turn into scientific notation const spotPrice = new BigNumber(assetABalance).dividedBy(assetBBalance).toString(); return spotPrice; } ``` --- ## Spot price as a field policy (2/6) We'll have to define a field policy for the `spotPrice` field. It's similiar to sub-resolvers, but strictly **synchronous** and cache oriented. We will: - Get the `id` of the current/parent entity using `readField` - Do a partial read on the cache a.k.a. read a fragment, in order to get the balances from our pool - We'll calculate the spot price using the function we've implemented earlier ```typescript= // src/lib/pool.tsx export const spotPriceFieldPolicy: FieldReadFunction<string | undefined> = (_spotPrice, { cache, readField, variables }) => { const id = `${__typename}:${readField('id')}`; const pool = cache.readFragment<Balances>({ id, variables, fragment: gql` fragment DataFragment on Pool { balances(assets: $assets) { assetId, free }, } `, }); if (pool && variables) { return calcualateSpotPrice(pool, variables['assets']) } } ``` --- ## Spot price as a field policy (3/6) Next step would be to register our field policy function with Apollo. It does not have any contextual dependencies so we can just throw it in: ```typescript= // src/hooks/useApolloClient.tsx export const useApolloClient = () => { const client = useMemo(() => { return new ApolloClient({ uri: '', cache: new InMemoryCache({ /** * */ typePolicies: { Pool: { fields: { spotPrice: spotPriceFieldPolicy } } } }), // you need at least an empty object here, // otherwise @client data aren't resolved resolvers: {} }) }, []); ... ``` --- ## Spot price as a field policy (4/6) Last step would be to update our query and the display component: ```typescript= // src/lib/pool.tsx export interface SpotPrice { spotPrice?: string } ``` ```typescript= // src/hooks/useGetPoolByAssetsQuery.tsx export const getPoolByAssetsQuery = gql` # Query that looks for a pool by a tuple of assets # Note: GraphQL doesnt have tuples, just arrays, # but country girls make do 🌽 query GetPoolByAssets($assets: [String!]!) { # ⚠️ pass the parameter to the resolver here pool(assets: $assets) @client{ id, assets, balances(assets: $assets) { assetId, free }, spotPrice } } ` // we need to know what kind of data does the query return export interface GetPoolByAssetsQueryResponse { pool: Awaited<ReturnType<ReturnType<typeof getPoolById>>> & Balances & SpotPrice } ``` --- ## Spot price as a field policy (5/6) ```typescript= // src/components/Pool.tsx export const Pool = ({ pool, loading }: PoolProps) => { return <> {!loading ? pool ? <> <p>ID: {pool.id}</p> <p>Assets: {pool.assets[0]} / {pool.assets[1]}</p> <div> <p>Balances:</p> {pool.balances.map(balance => ( <div key={balance.assetId}> <p>AssetId: {balance.assetId}</p> <p>Free balance: {balance.free}</p> </div> ))} </div> <p>Spot price: {pool.spotPrice || '???'}</p> </> : 'No pool found :/' : 'loading...' } </> } ``` --- ## Spot price as a field policy (6/6) ![](https://i.imgur.com/A3C5nlF.png) ---  <center> # Data freshness ![](https://i.imgur.com/07OK3cb.gif) </center> --- ## Blockchain time-keeping (1/*) Blockchain in general have two concepts of 'time': - block count - actual time / timestamp In our case, we can't be certain if the data that we're displaying is still relevant. There are two approaches that we could apply to make sure we have fresh data: - Subscriptions - Polkadot.js SDK offers subscription for chain state queries, they'll trigger a callback when new data is available - Invalidating cached data with new blocks - This builds on the assumption that data *may* change with every block, while keeping the data flow relatively straight forward We'll take a closer look at cache invalidation next. --- ## Blockchain time-keeping (1/*) Invalidating the cache isn't sufficient on it's own, we also need to **refetch queries** that have been previously cached and are now invaldiated (or their data). We'll implement a simple function that listents to new blocks on the chain, and once there is a new block it invalidates *all* cached queries. This will trigger a data refetch - and we're good to go. ```typescript= // src/lib/invalidateCachePerBlock.tsx export const invalidateCachePerBlock = async (client: ApolloClient<NormalizedCacheObject>, apiInstance: ApiPromise) => { return await apiInstance.derive.chain.bestNumber(() => { client.refetchQueries({ include: 'all' }); }) } ``` ```typescript= // src/hooks/useApolloClient.tsx const unsubscribeInvalidation = useRef<VoidFn>() useEffect(() => { (async () => { if (!dependencies?.polkadotJs) return; unsubscribeInvalidation.current = await invalidateCachePerBlock(client, dependencies.polkadotJs); })() return () => unsubscribeInvalidation.current && unsubscribeInvalidation.current(); }, [client, dependencies]) ``` ---  <center> # That's all folks.. or is it? ![](https://i.imgur.com/fjPQfZY.gif) </center> --- TODO: buy OR sell with tradeLimit + simple wallet integration + transaction status handling ---  <center> # #5 Historical data ![](https://i.imgur.com/bQTqO5k.gif) </center> --- ## Nodes are not enough for scalable web3 dApps Querying the blockchain node through its RPC for historical data is not a scaleable solution for Web3 dApps. * storage calls through RPC are slow * especially when done for hundreds of data points * some nodes do not hold history for long * resulting in even higher loads for those archive nodes Solution is a thin backend for historical data fetching. --- ## Web3 backend with Subsquid Subsquid is a framework to index the blockchain state, process the data specific to the project's needs and serve it through robust APIs. The two main components for doing this is an **indexer** that queries the blockchain node for all emitted events & extrinsics and a **processor** that only requests a subset of data, which is transformed and later served to the web3 dApp. In other words, an indexer is specific to one blockchain, whereas multiple processors can connect to the same indexer to retrieve and process the blockchain state. In this workshop we will write our own processor. ---  <center> ![](https://i.imgur.com/LRuO0w8.png) </center> *Archive aka Indexer --- ## From extrensic to processed events ```sequence User->Node: submits extrinsic Note over Node: update state, \n emit event(s) Indexer -> Node: query all \n events & extrinsics Node -> Indexer: Note over Indexer: save to DB \n serve through GraphQL Processor -> Indexer: query specific events Indexer -> Processor: Note over Processor: transform data \n save to DB \n serve through GraphQL ``` --- ## Our playbook for this chapter 1. Define Schema 2. Identify events and define processing strategy 3. Implement the processor 4. Explore data with GraphiQL --- ## 1. Defining the Schema ```graphql # schema.graphql type Pool @entity { id: ID! # pool address assetA: BigInt! assetB: BigInt! assetABalance: BigInt! assetBBalance: BigInt! # historicalBalances - comes later } ``` --- ## Events through the lens of polkadot.js ![](https://i.imgur.com/LIYc6oO.png) --- ![](https://i.imgur.com/QcgxKhl.png) --- ## What the indexer saves for `xyk.PoolCreated` ```json { "data": { "substrate_event": [ { "method": "PoolCreated", "blockNumber": 28, "params": [ { "name": "param0", "type": "AccountId32", "value": "bXmPf7DcVmFuHEmzH3UX8t6AUkfNQW8pnTeXGhFhqbfngjAak" }, { "name": "param1", "type": "u32", "value": 0 }, { "name": "param2", "type": "u32", "value": 2 }, { "name": "param3", "type": "u128", "value": "0x00000000000000000de0b6b3a7640000" }, { "name": "param4", "type": "u32", "value": 3 }, { "name": "param5", "type": "AccountId32", "value": "bXjT2D2cuxUuP2JzddMxYusg4cKo3wENje5Xdk3jbNwtRvStq" } ] } ] } } ``` --- ## Digging deeper into RUST code [XYK Pallet](https://github.com/galacticcouncil/Basilisk-node/blob/v7.0.0/pallets/xyk/src/lib.rs#L204) ```rust // pallets/xyk/src/lib.rs ... pub enum Event<T: Config> { /// Pool was created. [who, asset a, asset b, initial shares amount, share token, pool account id] PoolCreated(T::AccountId, AssetId, AssetId, Balance, AssetId, T::AccountId), ... } pub fn create_pool( origin: OriginFor<T>, asset_a: AssetId, asset_b: AssetId, amount: Balance, initial_price: Price, ) -> DispatchResult { ... Self::deposit_event(Event::PoolCreated( who.clone(), asset_a, asset_b, shares_added, share_token, pair_account.clone(), )); T::Currency::transfer(asset_a, &who, &pair_account, amount)?; T::Currency::transfer(asset_b, &who, &pair_account, asset_b_amount)?; ... } ``` --- ## Our processing strategy ![](https://i.imgur.com/rwVIU0u.png) --- ## Defining events in the processor ```json # typegen.json { "events": [ "balances.Transfer", "tokens.Transfer", "xyk.PoolCreated" ], ... ``` --- ## Getting started with subsquid ```typescript= // processor.ts import { SubstrateProcessor, } from '@subsquid/substrate-processor'; const processor = new SubstrateProcessor('amsterDOT-workshop'); processor.setBatchSize(500); processor.setDataSource({ archive: 'http://amsterdot-archive.eu.ngrok.io', chain: 'wss://amsterdot.eu.ngrok.io', }); ``` --- ## Adding event handler for xyk.PoolCreated ```typescript= // processor.ts processor.addEventHandler( 'xyk.PoolCreated', async (ctx: EventHandlerContext) => { const xykCreatedEvent = new XykPoolCreatedEvent(ctx); const parameters = xykCreatedEvent.asLatest; ... } ); ``` --- ## EventHandlerContext ```typescript export interface EventHandlerContext { store: Store block: SubstrateBlock event: SubstrateEvent extrinsic?: SubstrateExtrinsic } ``` ```typescript= // generated with make typegen export class XykPoolCreatedEvent { constructor(ctx: EventHandlerContext) ... /** * Pool was created. [who, asset a, asset b, initial shares amount, share token, pool account id] */ get asV40(): [v40.AccountId32, number, number, bigint, number, v40.AccountId32] { assert(this.isV40) return this.ctx._chain.decodeEvent(this.ctx.event) } } ``` --- ## Pool Schema in more depth ```graphql type Pool @entity { id: ID! # pool's address assetA: BigInt! assetB: BigInt! assetABalance: BigInt! assetBBalance: BigInt! } ``` With `make codegen` the following model is generated: ```typescript @Entity_() export class Pool { constructor(props?: Partial<Pool>) { Object.assign(this, props) } /** * ID is Pool address */ @PrimaryColumn_() id!: string @Column_("numeric", {transformer: marshal.bigintTransformer, nullable: false}) assetA!: bigint @Column_("numeric", {transformer: marshal.bigintTransformer, nullable: false}) assetB!: bigint @Column_("numeric", {transformer: marshal.bigintTransformer, nullable: false}) assetABalance!: bigint @Column_("numeric", {transformer: marshal.bigintTransformer, nullable: false}) assetBBalance!: bigint } ``` --- ```typescript= processor.addEventHandler( 'xyk.PoolCreated', async (ctx: EventHandlerContext) => { // xykCreatedEvent = [who, asset a, asset b, initial shares amount, share token, pool account id] const xykCreatedEvent = new XykPoolCreatedEvent(ctx); const xykPoolId = ss58 .codec('basilisk') .encode(xykCreatedEvent.asLatest[5]); const pool = await getOrCreate(ctx.store, Pool, xykPoolId); pool.assetA = BigInt(xykCreatedEvent.asLatest[1]); pool.assetB = BigInt(xykCreatedEvent.asLatest[2]); pool.assetABalance = BigInt(0); pool.assetBBalance = BigInt(0); await ctx.store.save(pool); console.log('new xyk pool saved with id:', xykPoolId); } ); ``` --- ## Spin up the processor and the query-node ``` yarn run processor:db yarn run processor:reset yarn run processor:start # in another window yarn run query-node:start ``` --- ## Querying the processor's db Navigate to `https://graphiql-online.com/graphiql` and use `https://amsterdot-processor.eu.ngrok.io/graphql`. ```typescript query ExampleQuery { pools { id assetABalance assetBBalance assetA assetB } } ``` ```json { "data": { "pools": [ { "id": "bXn6KCrv8k2JV7B2c5jzLttBDqL4BurPCTcLa3NQk5SWDVXCJ", "assetABalance": "0", "assetBBalance": "0", "assetA": "0", "assetB": "1" } ] } } ``` ---  <center> # Historical data in the UI ![](https://i.imgur.com/6pDu5eG.gif) </center> --- ## Consuming external data sources (1/*) Thanks to Apollo Client, we can combine `@client` and remote data sources quite easily - even within the same query. First thing we have to do is make sure Apollo knows what GraphQL server we want to talk to: ```typescript= // src/hooks/useApolloClient.tsx export const useApolloClient = (): ApolloClient<NormalizedCacheObject> => { const client = useMemo(() => { return new ApolloClient({ uri: 'https://amsterdot-processor.eu.ngrok.io', ... ``` --- ## Consuming external data sources (1/*) # Links - Code - [Basilisk Node](https://) - [UI](https://) - [Processor](https://) - [Workshop](https://github.com/galacticcouncil/amsterdot-ui-workshop) - Docs - [GraphQL](https://) - [Apollo Client](https://) - [React](https://) - [Create React App](https://) - Tools - [Apollo Chrome Dev Tools](https://) - [VS Code Apollo Client](https://)