Lightning

Released: Tuesday, April 11th 11:59pm EST
Due: Tuesday, April 25th 11:59pm EST

Introduction

Now that we're familiar with a blockchain's Layer 1 from Coin, it's time that we move on to a Layer 2 scaling solution! In class, you've learned about off-chain payment channels. Lightning is a scaling protocol for Bitcoin, and we've done our best to imitate that here.

While we did cover Lightning (very) briefly in class, we strongly recommend that you watch the following 20 min video to better understand what we're asking you to do. Without a strong conceptual understanding, this project will be a toughie. And that would suck :(

Components

1. Node
   • We already learned about nodes in Coin, but we'll be making a few updates with this assignment. First, we'll be implementing the SegWit protocol, which allows us to fit more transactions in each block. Second, we'll be accomodating our new Lightning node!
2. LightningNode
   • Like our normal nodes, lightning nodes run their own server, deal with transactions, and have a collection of other (lightning) peers. Unlike our normal nodes, lightning nodes keep track of channels, which keep track of transactions without having to broadcast everything to the main network.
3. Channel
   • Once a channel is opened between two parties and backed by a transaction on the main network, channel members can send updated transactions back and forth without having to broadcast all minor updates to the main chain. Think of all the pesky fees and time saved! When one party decides to "settle," they simply broadcast the most recent transaction to the main network.
4. Watchtower
   • Since every transaction in a channel is technically valid, we have to employ the services of a watchtower to ensure that our counterparty doesn't cheat and broadcast an outdated transaction! The watchtower will keep a collection of all revocation keys for past transactions, ensuring that if the counterparty does decide to cheat, they'll be caught (and lose all of their coins in the process)!
5. Server
   • You'll have to deal with two servers for this assignment! First, you'll implement the SegWit protocol on the node's server. Then you'll implement the lightning protocol on the lightning node's server.

Assignment

For this assignment, you will implement the functions listed below. We recommend that you tackle the functions in the following order:

SegWit

1. pkg/node.go: BroadcastTransaction
2. pkg/server.go: GetWitnesses
3. pkg/server.go: ForwardTransaction

Wallet

4. pkg/wallet.go: GenerateFundingTransaction
5. pkg/wallet.go: HandleRevokedOutput

Lightning Server

6. lightning/server.go: OpenChannel
7. lightning/server.go: GetUpdatedTransactions
8. lightning/server.go: GetRevocationKey

Lightning

9. lightning/channel.go: CreateChannel
10. lightning/lightningnode.go: UpdateState
11. lightning/watchtower.go: HandleBlock

To see how difficult we think each function is, check out our grading breakdown.

SegWit

The segregated witness protocol is one of the most beloved soft forks in Bitcoin. By segregating the witnesses from the transactions, miners can squeeze more transactions into each block. While they may or may not receive more in fees (long-term equilibrium anyone?), it certainly increases the throughput of the network. It's estimated that signatures take up ~65% of a block's capacity. By removing signatures from blocks, Bitcoin's scalability signifcantly improves!

This part of the assignment should be relatively quick: it's only updating a few lines of code here and there.

Stencil update:
We've added two new fields to transactions: segwit and witnesses. Segwit is a boolean flag letting a node know whether this transaction is employing the segregated witness protocol. Witnesses are a list of signatures on a transaction.

Unlike most normal hashing methods (which hash all of the fields), our transaction hashes do NOT include the slice of witnesses. Instead, they omit them, allowing signatories to see a transaction and later add their signature, WITHOUT altering the transaction's hash.

Before:

type Transaction struct {
	Version   uint32
	Inputs    []*TransactionInput
	Outputs   []*TransactionOutput
	LockTime  uint32
}

Now:

type Transaction struct {
	Segwit    bool
	Version   uint32
	Inputs    []*TransactionInput
	Outputs   []*TransactionOutput
	Witnesses [][]byte
	LockTime  uint32
}

A Note on context.Context:

You do NOT need to worry about context for this assignment. Whenever you see ctx context.Context as a method parameter for a server-side function, don't worry about it!

Implement:

// BroadcastTransaction broadcasts transactions created by the 
// wallet to other peers in the network.
func (n *Node) BroadcastTransaction(tx *block.Transaction)

Overview:

• This is a quick one: all you have to do is remove the witnesses from the transaction before you send it off to your peers. You should remove the witnesses AFTER you've encoded the transaction, so that you don't remove the witnesses of our transaction. Other nodes following the SegWit protocol will request this data from you later.

Implement:

// GetWitnesses is called by another SegWit node to get the witnesses (signatures) from you.
func (n *Node) GetWitnesses(ctx context.Context, in *pro.Transaction) (*pro.Witnesses, error)

Overview:

• Another quick one. If your node is serving this request, it means that another node wants the witnesses from you!
• You have theses witnesses stored in your SeenTransactions mapping, so it's just a matter of sending those signatures over.
• Check out the other functions in this file to see how you should send over the witness data! It has to match the formatting of our pro.Witnesses struct.

Helpful Functions

• status.Errorf(codes.Internal, "Your error message"): Since this is an RPC, we use the status package when handling errors.

Implement:

// ForwardTransaction Handles forward transaction request (tx propagation)
func (n *Node) ForwardTransaction(ctx context.Context, in *pro.TransactionWithAddress) (*pro.Empty, error)

Overview:

• You actually may have seen this function from the last project, Coin. We have to update it to deal with the SegWit protocol.
• If we've seen their transaction before, we just update this transaction's count in our SeenTransactions and return
• If we haven't seen this transaction before, we'll need to check if the transaction we're being requested to forward follows the SegWit protocol.
• If the new transaction does follow the SegWit protocol, we need to get the witnesses for it! So, we ask the message's sender to pass along the witnesses, and set them for our transaction.
• Finally, we add the new transaction to our SeenTransactions with a count of 1.

Helpful Functions:

• func (a *Address) GetWitnessesRPC(request *pro.Transaction) (*pro.Witnesses, error): for when we need the sender to get us the witnesses!

Wallet

We were introduced to the wallet in the last project, but now it has to form transactions for Lightning channels!

LockingScripts:
Bitcoin has its very own scripting language, which permits the creation of coins that have to be signed by multiple people. While we didn't create our own scripting language, we did our best to emulate Bitcoin Script with our 3 different types of locking scripts in pkg/script/lockingscript.go.

PayToPublicKey: this is the most common type of lockingScript. If I want to send a coin to someone, they only need to prove that they are the owner of the public key to use it (validation requires their private key).

MultiParty: These are the types of scripts that we will be making use of for our Lightning channels. They contain the public keys for both members of a channel, as well as a revocation key and a number of additional blocks. No one can use the TransactionOutput unless both parties have signed off on the Transaction as witnesses!

HashedTimeLock: We (unfortunately) won't be using these for this assignment, but this is how we can create hops between multiple nodes and truly take advantage of the Lightning network!

Implement:

// GenerateFundingTransaction is very similar to RequestTransaction, except it does NOT broadcast to the node.
// Also, the outputs are slightly different.
func (w *Wallet) GenerateFundingTransaction(amount uint32, fee uint32, counterparty []byte) *block.Transaction

Overview:

• The fee passed in to the function is actually worth 2 fees! Half is to cover the funding transaction, and the other half is to cover the refund transaction (which the Lightning node handles later on).
• When generating the inputs, make sure that you set the amount field as amount + fee. This is because we need to provide enough for both the funding and the refund transaction.
• Now, we need to generate our "lightning" outputs. Unfortunately, generateTransactionOutputs Won't work for this.
• Lightning transactions have three outputs. The first is the coin for the lightning channel's funder, which has an amount of amount. The second is the coin for the counterparty, which always has an amount of 0. If there is a third output, it's change that goes back to the channel's funder (since it's using their coins) once the channel is closed. Change needs to include the fee for the refund transaction.
• We also need to create a lockingScript for all of the coins. We'll use a MultiParty locking script to do so. In this case, there is no revocation key, so you can leave that field as []byte{}. This script enforces that both parties have signed off on the funding transaction.

Helpful Functions:

• proto.Marshal(m message) ([]byte, error): how to convert a pro.MultiParty to a byte slice.
• generateTransactionInputs: to create the inputs for our transaction

Implement:

// ForwardTransaction Handles forward transaction request (tx propagation)
// HandleRevokedOutput returns true if it successfully handles revoking
// the transaction
func (w *Wallet) HandleRevokedOutput(hash string, txo *block.TransactionOutput, outIndex uint32, secRevKey []byte, scriptType int) *block.Transaction

Overview:

• We first need to check whether the secret revocation key is the private key for the public revocation key in the lockingScript. If it isn't, we can just return nil
• We should make the output we want to revoke into an input for a new transaction. That's how we'll claim it.
• Since we're just paying ourselves, our lockingScript should be the byte version of a PayToPublicKey with our public key.
• Now that we have our lockingScript, we should create a new transaction output using that script and the amount of the revoked coin. We need to provide enough for the (implicit) fee, which can be found in w.Config.DefaultFee.
• Now that we have our new input and new output, we can make the transaction, sign it, and broadcast it to the node!

Helpful Functions:

• RevKeySuccessful(lockingScript []byte, secRevKey []byte, scriptType int) bool: returns whether the secret revocation key validates the lockingScript
• proto.Marshal(m message) ([]byte, error): how to convert a pro.PayToPublicKey to a byte slice.
• utils.Sign(sk *ecdsa.PrivateKey, h []byte) ([]byte, error) : how to sign something. A transaction's hash is a string, which we can quickly convert to bytes.

Lightning Server

A Lightning node runs its own server, which is how it receives requests from other lightning nodes and responds to them. You'll be implementing the server-side responsibilities of a lightning node: handling open channel requests, updated transaction requests, and revocation key requests.

Updating State Protocol:

Both Alice and Bob have their own versions of each state, since the revocable outputs are different for each side. Tx(1,A) is Alice's transaction for state 1. Rev(3,B) Is Bob's revocation key for his transaction at state 3.

In the example below, Alice and Bob have an existing channel set up. Alice wants to pay Bob, so she has to be the one to initiate the protocol. If Bob wanted to pay Alice, the process would be inverted.

Implement:

// OpenChannel is called by another lightning node that wants to open a channel with us
func (ln *LightningNode) OpenChannel(ctx context.Context, in *pro.OpenChannelRequest) (*pro.OpenChannelResponse, error)

Overview:

• Before any exchange can happen like the one above, two lightning nodes need to establish a channel!
• This function is a lightning node's RESPONSE to an open channel request from another node.
• First, we'll need to make sure that we know about the request's sender. If we do, they'll be in our PeerDb. If they aren't, we should stop this function's execution early and return.
• Next, we check if we already have a channel set up with this peer. If we do, we should return.
• We should validate and sign off on both the funding and refund transactions that we've been sent.
• Now it's time to open a channel with this peer! Since our coin is worth 0 on the refund transaction, we can add it to both TheirTransactions and MyTransactions. That won't be the case in the future!
• Likewise, we can generate a dummy revocation key for this transacation and add it to MyRevocationKeys, which we'll send over when we want to update state later.
• Now it's time to our response back to the other node, which include our public key and both signed transactions.

Helpful Functions:

• PeerDb.Get(string) *Peer: returns nil if the address is not in our PeerDb.
• func (ln *LightningNode) ValidateAndSign(tx *block.Transaction) error: how to validate and sign a transaction
• GenerateRevocationKey(): generates a public, private key pair (in byte form)

Implement:

func (ln *LightningNode) GetUpdatedTransactions(ctx context.Context, in *pro.TransactionWithAddress) (*pro.UpdatedTransactions, error)

Overview:

• Now that we have a channel set up, it's time for us to imitate the protocol above! Since we're the server side, we're taking on Bob's role for this one.
• Like when opening a channel, we should validate the address we've been sent.
• We sign the transaction we've been sent, and add our signature to the slice of witnesses. Now that our coin is worth something, we have to make our own version of this transaction where the counterparty's coin is revocable. Before we do so, we'll have to generate a new revocation key pair.
• Now we can make our version of the transaction, where their coin is revocable.
• We can add their newly signed transaction to TheirTransactions and the secret key we just made to MyRevocationKeys.
• All that's left is to return their signed transaction and the transaction that we want them to sign for us!

Helpful Functions:

• func (ln *LightningNode) generateTransactionWithCorrectScripts(peer *peer.Peer, theirTx *block.Transaction, pubRevKey []byte) *block.Transaction: creates a transaction with the correct locking scripts.

Implement:

func (ln *LightningNode) GetRevocationKey(ctx context.Context, in *pro.SignedTransactionWithKey) (*pro.RevocationKey, error)

Overview:

• The last step of our protocol. As always, we should validate the address of the person we're receiving the request from.
• We can now add our newly signed transaction to MyTransactions.
• We now need to create a RevocationInfo that contains all of the information necessary to claim the counterparty's coin from the previous state.
• Before we create the RevocationInfo, we need to figure out what type of script they were using on their transaction and figure out which coin their revocation key belongs to.
• If we're the channel's funder, then our coin will be the first output. If not, it'll be the second output. We want to revoke the coin that isn't ours!
• We can now add the newly constructed RevocationInfo to TheirRevocationKeys.
• Lastly, we have to get our own revocation key for OUR version of that transaction.
• We can finally increment our state and send them our revocation key.

Helpful Functions:

• func DetermineScriptType(b []byte) (int, error): how to determine the type of a locking script

Lightning

Now we're taking on Alice's side of the protocol. An owner of a lightning node probably wants to establish a couple of channels, so that they have more opportunities to take part in multi-channel hops (not part of this assignment) and hopefully collect some routing fees along the way!

Updating State Protocol:

Here's the protocol from above, for easier reference.

Implement

// CreateChannel creates a channel with another lightning node
// fee must be enough to cover two transactions! You will get back change from first
func (ln *LightningNode) CreateChannel(peer *peer.Peer, theirPubKey []byte, amount uint32, fee uint32)

Overview:

• Our first step is to create a struct for the new channel. Then, we can add it to our channels mapping.
• We'll need to make a WalletRequest to the wallet, so that it can use valid coins to create the funding transaction for us. The fee here has to be twice the fee provided as an argument, to cover both the funding and refund transaction.
• Once we've received our funding transaction from the wallet, we have to make a revocation key pair for our refund transaction.
• We can make the refund transaction ourselves, since we have a fully formed funding transaction. Reminder: the refund transaction uses the outputs created by the funding transaction (which is on the chain) as inputs for itself.
• We should also add the private revocation key to MyRevocationKeys.
• Now that we have everything we need, it's time we request another lightning node to open a channel with us!
• Once we get our response (you coded this earlier!), we can set the channel's funding transaction and store the refund transaction as both party's first transaction. This is fine, since we don't need to be able to revoke the refund transaction.
• Now it's time for us to sign our own funding transaction and broadcast it to the node.
• Once we see the funding transaction–-with enough POW on top of it–-on the blockchain, we can start sending money back and forth in this channel.

Helpful Functions:

• func (ln *LightningNode) generateFundingTransaction(request WalletRequest) *block.Transaction: How to get a funding transaction from the wallet.
• func (ln *LightningNode) generateRefundTransaction(theirPubKey []byte, fundingTx *block.Transaction, fee uint32, revKey []byte) *block.Transaction: How to create a refund transaction given a funding transaction.
• func (a *Address) OpenChannelRPC(request *pro.OpenChannelRequest) (*pro.OpenChannelResponse, error): RPC call to request opening a channel with another lightning node.

Implement

// UpdateState is called to update the state of a channel.
func (ln *LightningNode) UpdateState(peer *peer.Peer, tx *block.Transaction) 

Overview:

• This call encompasses both of Alice's requests on the diagram above.
• First, we need to make the appropriate request and get the updated transactions from our peer.
• Then, we should add our signed transaction to MyTransactions, sign their transaction, and add their transaction to TheirTransactions.
• We then need to make another request to our peer to get their revocation key, this time sending them their signed version for the new state, along with our revocation key for the current state.
• Once we get their revocation key, we can increment our state! All parties have transactions for the updated state, and everyone can revoke the previous state.
• We now need to store their revocation key. This process is very similar to the server-side logic (which you coded above). We need to figure out what the type of the script is and which coin belongs to them. We can finally store a RevocationInfo in TheirRevocationKeys.

Helpful Functions:

• func (a *Address) GetUpdatedTransactionsRPC(request *pro.TransactionWithAddress) (*pro.UpdatedTransactions, error): making an RPC call on the address's server to get our transaction signed and their updated transaction.
• func (a *Address) GetRevocationKeyRPC(request *pro.SignedTransactionWithKey) (*pro.RevocationKey, error): making an RPC call on the address's server to get the revocation key for their transaction.

Implement

//HandleBlock handles a block and figures out if we need to revoke a transaction
func (w *WatchTower) HandleBlock(block *block.Block) *RevocationInfo

Overview:

• Another quick one! Almost there :)
• Given a block, our watchtower looks to make sure that none of the transactions are ones that we need to be watching for.
• If any of the transactions are ones we're monitoring, we send the corresponding RevocationInfo to our RevokedTransactions, which is monitored by the node. It will then pass along the info to the wallet, which will handle revoking the coin (you already wrote this functionality).

Testing

This assignment is autograded, and you are able to run our test suite as many time as you like. While you can run all of your tests on Gradescope, we have also provided the entire test suite locally. We are hoping that this change can help students get a little bit more experience using GoLand's debugger, as well as better understand the architecture of the project.

In addition, you should write your own tests when things aren't working! We have provided several helper functions in test/mocking_utils.go and test/testing_utils.go.

To test your project, cd into test and run go test. This will let you know which tests are failing, and why. It will likely be more convenient to run individual tests, which you can do using the GoLand UI.

Note: most of our tests will cause the debugger to panic because they rely on functions that have not yet been implemented.

Install

1. Clone the stencil repo.
2. In your local Lightning directory, run go get ./... to install the project's dependencies.
3. Get after it! Good luck

HandIn

1. Log into Gradescope.
2. Submit your code using GitHub.

Grading

Note: Because of the new testing methodology we are taking with Lightning, this assignment's feedback form is a REQUIRED part of the assignment, worth 5 points. We want to understand how students prefer testing to look, as well as how different ways of testing can alleviate TA burden while maintaining educational and academic integrity.

We will also be performing some data analytics to see how big of a difference this change makes in terms of project completion time, average grade, etc…

The feedback form can be found at this link.

This assignment is out of 100 points. Your grade will be determined by completing the feedback form and passing the following test functions:

Rubric

Test Function	Points
TestGetWitnessesRPC	5
TestForwardTransaction	5
TestGenerateFundingTransaction	10
TestHandleRevokedTransaction	10
TestOpenChannel	10
TestGetUpdatedTransactions	15
TestGetRevocationKey	10
TestCreateChannel	10
TestUpdateState	15
TestWatchTowerHandleBlock	5
Feedback Form Completion	5
Total	100