# On further rationalizing the ROI metrics in the OCEANDAO

## Table Of Content

[Toc]

## Problem Statement

The oceanDAO mandates all proposals to publish a "Return-On-Investment" calculation along with their proposal [1]. The idea being that if only "positive ROI projects" get funding by the oceanDAO, then the oceanDAO will return an overall positive ROI. In the wiki, similarities to VC funding are mentioned [2].

Over the oceanDAO's past existence, however, this ROI metric has lead to numerous misunderstandings and conflicts within the DAO. In this document, I'd like to enumerate some of the problems that a grantee like rugpullindex has encountered with a heavy reliance on this rather theoretical and subjective metric framing.

## Insufficient distinction between _lag_ and _lead_ metrics

Generally speaking, when optimizing a process it's useful to track a variety of metrics that can serve as a proxy for the success of the optimization. However, to successfully navigate using a metric-driven development approach, it's advicable to distinguish metrics between _lag_ and _lead_ metrics:

- **Lag metrics**: Describe the thing you're ultimately trying to improve (e.g. the goal of increasing customer satisfaction or increase in sales/revenue)
- **Lead metrics**, in contrast, describe new behaviors that may drive success in lag metrics.

A good rule of thumb is that **lag metrics** *lag behind time*, while **lead metric** *lead the way*.

The oceanDAO ROI document, however, makes no distinction between these two types of metrics. It's separating between "primary" and "secondary" metrics, but those don't caputure the concept of lead and lag metrics.

I think, the oceanDAO ROI document should specifically make a conceptual distinction between different types of metrics and which are useful for tracking the near and/or long term.

## ROI calculation mandate promotes dubious practice of generating arbitrary positive ROI calculations

Is anyone aware of any oceanDAO proposer that, in the last second, withdrew their proposal because they've noticed that their ROI calculation won't turn positive?

In this case I'll just point out that clearly the tail wages the dog. Nobody would stop their project if their ROI calculation turned out negative. I think we can all agree that it wouldn't make much sense either. Instead, an author is expected to fiddle with the calculation until its return becomes non-negative.

Naturally, we can't expect any of the calculations to carry much signal regarding the actual return of the project. The calculation's existence is based on the idea to be non-negative and that's what their authors optimize it for.

## The data set market is emergent and it may not be useful to speculate on its near term value

To convince yourself of this statement, just look at the numbers of the OCEAN marketplace today.

If you actually do the research, you'll see that over the last 6 months there were hardly any data set consumptions. I'd argue that the total amount of data consumption remains in the five figure dollar range (it's only a guess). Most charts on rugpullindex are correlated heavily with OCEAN - because there's simply no trading happening on these data pools. So speculation isn't happening either yet.

Of course, none of this is meant in a bad way. It's merely addressing today's reality. If e.g. we at rugpullindex wanted to show a purely rationally-calculated ROI based on "network revenue" generated through users clicking from our website to buy on the OCEAN market - our *bang* would most likely be lower than the *buck*. Not because we did a bad job, but because the timing of our product and the timing of OPF's marketplace simply hasn't fully aligned with reality yet. Simply put: full product/market fit hasn't been achieved yet.

So then, why do a bang/buck calculation if bang is necessarily smaller than buck?

An example based on Evotegra. For 21 EXCANE-93 at a price of 980.27€, they've achieved a total value locked of 20k EUR which would only allow the participation of e.g. a single round in the oceanDAO. Obviously though, they've generated much greater bang than that, but applying the ROI framework yields no useful calculations.

Even if the Ocean Marketplace team were to propose in the DAO with a ROI calculation, it's likely they should be rejected on its basis too. Dont' get me wrong, they're arguably doing a great job building things. But they neither have managed to capture a *bang* that is high enough to justify the monthly *buck*. It's because we have to assume that most value will be captured in the future and over a longer period of time than just today.

So clearly, we're talking about returns of investment on a much longer scale, e.g. that through all the work done now - maybe in 5 years a ROI will become clearly positive.

## oceanDAO teams aren't in control when it comes to measuring

The primary fundamental metrics are "total ocean consumed", "network revenue" and "total value locked". However, to measure these metrics rationally and without some bogus assumption in their ROI calculations, oceanDAO teams require infrastructure that puts them in control when measuring.

In the case of RPI, since we've started tracking our outbound clicks, we've sent more than 500 people to promising data sets on the OCEAN marketplace. Undoubtably, some investors have taken the information provided on the website to further their investment decisions. There might have been some buys! But they don't show up anywhere and RPI isn't aware of them.

This is because it's very challenging for RPI to rationally track which clicks actually facilitated a sale. Given that the Ocean Marketplace has no way of reporting back to RPI when a sale has occurred, we're unable to know what happens on their site.

Personally, I doubt this problem can be addressed in the short term as building a sales referrer system is not a trivial task either. I've [submitted](https://github.com/oceanprotocol/market/issues/811) an issue on the marketplace for the time being.

What is becoming clear from this situation, however, is that RPI isn't in direct control of delivering any of the required metrics as we can't speed up or prioritize work on the Ocean Marketplace single handedly. We're not in control. For now, we're left to speculate and that's the opposite of rationalization.

## oceanDAO teams lack vital legal support structure for serving the required metrics

Another option that a project can take to rationalize their ROI is by e.g. directly spinning up a data token pool and measuring e.g. the amount of OCEAN staked. One example of this is DataUnion's data token pool.

But also with this, there are several hurdles that will first have to be overcome by teams before any ROI measurements:

- The project will have to spend money on a lawyer that can advise them to set up shop in a crypto-friendly juristiction
- The project will have to either relocate or be willing to take the potential risk of still being liable in their juristiction.
- The project will have to advertise the token to a degree that *bang* > *buck*.

But then also the question is: If the oceanDAO team is capable of spinning up their own token legally and advertise it - why ask for funds from the oceanDAO in the first place as selling their token might be a more value-capturing activity?

## Arbitrariness in calculation of ROI leads to continous controversy

As a continous receiver of funds from the oceanDAO since round 2, I've probably read more proposals than is good for me. One situation that I believe will lead to further continous controversy is the arbitrariness in calculation of ROIs.

There's simply no authority for ROI calculation that stands out. Everyone is on the side lines and the ROI document itself isn't giving clear guidance either.

E.g., there's not a common understanding of probability. It's commonly understood that like 90% of all startups will eventually fail and that their value will be zero or negative. Then why do many proposers state a `chanceOfSuccess` rate of much more than 10%?

There's a vastly different perception of what is truly valuable and what's not. E.g. in one of the last rounds there were proposals that calculated ROI's for newsletters in the range of more than $100000.

While I tend to agree that given an enormous success, a news letter could yield more than $100k, I think popular examples are vast outliers in a pool of mainly valueless news letters.

But hey, I don't doubt that these efforts weren't done with best intentions in mind. But then the second problem is that in case the project returns in a secondary round, at that point: How can we now rationally update our ROI calculation?

The worst part of this is: If controversy emerges, it's unclear which party is "right" or doing things in the proper way. It's because there's no proper way.

## Monthly ROI reviews haven't helped me

As I've said earlier, in the beginning of a project's lifecycle, calculating the business' ROI is definitely a helpful exercise in understanding where its value may sit. However, having to show up with an updated ROI calculation on a monthly basis hasn't provided any value and mainly frustration to me personally.

Between R2 and R8, not many of the fundamentals of rugpullindex have changed. Amongst other things, we still want to eventually launch an index token that allows users to buy exposure to data sets.

So between R2 and R8: What has changed that we could report on?

- We've started tracking our web analytics
- We've started tracking our customers of the API
- We're starting to track other metrics
- We've managed to continously ship

However, one could argue that none of these things influence either bang, buck or chanceOfSuccess. We're not tracking anything related to network revenue or other fundamental OCEAN metrics as simply we can't.

For the most time, we've logically resorted to building product as this has arguably lead to a positive outcome in the oceanDAO votes.

There'll definitely be the day where we want to launch the index token to our users. On that day, we'd like to publicly track e.g. network revenue to TVL. But until that day comes, somehow it should be possible for us to make a rational case to the oceanDAO for why OCEAN holders should vote for our product.

We'd appreciate if the oceanDAO appreciated to measure REAL and existing metrics *first*.

I'd like to make clear that I'm not "against" the ROI calculation. Rather I'm arguing FOR AN IMPROVED concept of tracking the metrics within the oceanDAO. I think distinguishing between types of metrics can help. Further education and deliberate moderation can help too. I just would like to avoid continuing with the status quo as it's frustrating for many.

## References

- 1: https://github.com/oceanprotocol/oceandao/wiki/Grant-Proposal-Template
- 2: https://github.com/oceanprotocol/oceandao/wiki/On-ROI#similarity-to-venture-capital
- 3: NEWPORT, Cal. Deep work: Rules for focused success in a distracted world. Hachette UK, 2016.