# Software 2.0 - Systems of utility

> Last week when Klarna announced that they were [cutting off Salesforce and Workday](https://www.inc.com/sam-blum/klarna-plans-to-shut-down-saas-providers-and-replace-them-with-ai.html) and replacing them with internal AI built applications, but this trend will only magnify and amplify itself over the next 3-5 years where purely horizontal "systems of record" will find themselves replaced almost completely.

The average spend on SAAS per employee is [$8,700 per annum](https://www.vertice.one/l/saas-inflation-index-report). At some point, we must look back and ask how much more value add is this when OpenAI team costs $500 per annum? How much of the system is a force mutliplier and how much of it is bookkeeping?

## Business Software Sucks
A lot of the business software was literally built as a custom dashboard on top of an excel(or postgres)-like source of truth where you could add/edit/update/delete information according to a specific data model and (mostly role-based) access control schema. Obviously this was layered with a million bells and whistles across screens.
\
This meant that the software was shipped more to convince the (mostly business) buyer that it has everything in place to make sure that crucial business data wasnt lost, misplaced, or misused. Anything beyond that was fulfilled by the 1000s of vendors and integration partners on their marketplace (like ZoomInfo for Salesforce or literally all of Shopify). **This is literally bloatware**
\
This fragmentation was very frustrating because the cost of software in action far exceeded that of the annual SAAS subscription itself. This is how the traditional cost split looked like for implementing salesforce within a team of 20-100 people.
\
<img src="https://hackmd.io/_uploads/S1OtmlLT0.png" height="450" style="margin-left:30%;">

Many companies tried to fix this bloat using **internal tool builders**(Retool), **BI platforms**(Tableau), and **workflow builders**(Zapier) but the incentives for doing so long-term were not aligned for star performers on building great internal tools.

## SAAS 2.0 - The Hype Cycle
At the very beginning of the ChatGPT hype cycle you had people claiming that all software was going to be replaced by ChatGPT with document upload. Obviously this wasnt how the best RAG systems work but this not directionally wrong as we can see from Salesforce's (the biggest system of record) AI investments.
\
The most famous/extreme usage of this term is by Andrej Karpathy where we have a computer that does not run any programs at all, just supports a neural net which converts input streams into output streams.
<br>
<blockquote class="twitter-tweet" data-theme="dark"><p lang="en" dir="ltr">100% Fully Software 2.0 computer. Just a single neural net and no classical software at all. Device inputs (audio video, touch etc) directly feed into a neural net, the outputs of it directly display as audio/video on speaker/screen, that’s it.</p>&mdash; Andrej Karpathy (@karpathy) <a href="https://twitter.com/karpathy/status/1807497426816946333?ref_src=twsrc%5Etfw">June 30, 2024</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script><br>

Of course GPT4 sent Twitter mad about this, with VCs enthusiastically proclaiming either one of the 2 things:
1. Large companies are moving too fast and they will capture this trend entirely with native Gen-AI products.
2. Large companies are being disrupted entirely by a new data model in very fundamental spaces and there is a time for disruption.

## SAAS 2.0 - How to Build One?
Does adding a chat window to your product make it SAAS 2.0? If not, the question must be then what does? And there are product level answers and a pricing level answers.
\
In short SAAS 2.0 resolves moving from **systems of record** to [**systems of utility**](https://www.ycombinator.com/companies?industry=B2B&tags=Generative%20AI). Data present in your software has to be inherently valuable just like how ChatGPT is a queriable and condensed version of all the internet.
\
On the pricing level, the answer is very simple, if your cost structure looks something like the left hand side of the below image, OpenAI is going to replace you and if it looks like something on the right then it is a sustainable SAAS 2.0 product. Similar to how an app like Datadog or Salesforce would only pay out 5-10% of their revenue on cloud infrastructure.
\
<img src="https://hackmd.io/_uploads/r1F1bJvpC.png" height="430px">
\
On the product side its slightly more complicated but a simple one liner can be: **Is more than 50% of the value of the product coming from things other than being a system of record and are those features AI orchestrated or generated?**

### For product builders, here is a more detailed checklist of capabilities:
- [ ] Is your product able to do migrations and data level maintenance automatically?
- [ ] Does your product have enough business context to close the loop with any information that may be recorded without needing dedicated integrations?
- [ ] Can it dynamically support new user workflows based on PRD and existing API schema?
- [ ] Is your product able to understand user persona and mould the user journey accordingly?
- [ ] Does your product automatically optimize the data model based on usage pattern or customer-level intricacies?

In the next section, I will be going through the agent architecture corresponding to each of these 5 items along with the pricing/revenue ROI you should expect once each of them are deeply integrated.

## Level 1: Context Owner
The lowest hanging fruit, is an AI agent that can understand new information being captured and fetch relevant background information for the same. This is the #1 thing you can do increase your SAAS's stickiness.
\
The simplest implementation would be to map the employee who normally enter information into your SAAS and have the LLM replicate their behavior to extract the actionable insights from the new piece of information.
\
Once the new information is extracted, you can order the actionables (preferrably into a DAG) and start an agentic search process for each actionable over the company wiki (or any other source of truth) and harmonize all fetched context. Then we can pass all the tagged context information to whichever system needs it.

### Agent Architecture
<img src="https://hackmd.io/_uploads/BJC74Xup0.png" height="720px">

\
This agent's output needs to be open ended so that we can reuse the context around new information across multiple systems with 1 LLM call for parsing.
\
If you are able to implement this agent using a LLaMa 3.1 70B sized model, you will be paying a maximum of $100 monthly per enterprise customer of your product.

## Level 2: Data Model Orchestrator
Now lets go one level further. Say you have some information that needs to changed (could be manually or via integrations or with automations) how do you make sure that each person doing so doesnt have to waste time separately on verifying information, sanity checks around data and so on?

The easiest answer (only if you have a good metadata store), might be to do some generative fill on data using the model's IQ (and reading the DB) along with a second LLM call to parse the filled output into your relevant schema.

Then we can pass it into a further call to order it into insert and delete transactions based on the DB index. The final LLM call will generate sanity checks which can be run both pre and post data insertions.

### Agent Architecture
<img src="https://hackmd.io/_uploads/B17SZevTR.png" height="465px">
<br><br>

> **Note 1:** Removing the possibility of update or read transactions greatly improves consistency in model output

<br>

> **Note 2:** Implementing the sniff tests as stored procedures ensures that any checks failing can be handled gracefully without needing custom code using the database's native transaction revert process.

## Level 3: Workflow Assistant
Now, we are getting into the slightly harder to implement features. The bare minimum requirement for this is to have an API schema for your platform that a gpt-4o-mini type of model can understand 80% of the time or more. ([PS: You can evaluate this here](gorilla.cs.berkeley.edu/leaderboard.html))
\
But the core idea is to replicate an advanced version of something like this by analyzing your user data and in response to a query that may have internal jargons or known workflows. For example, **"Export the client leads from Apollo for the financial compliance usecase into Salesforce"** on the below chatbox:
\

The 4 places which you will try and will in from your agent vs a single LLM call is to have control on:
1. Overall task planning w.r.t to employee permissions and usecase/business context
2. Optimal Tool Selection
3. Trigger Mechanism b/w steps
4. Glue Scripts for data transformation b/w steps

**For example in the following tasks:**
|Task Details| Standard Automation Agents | Workflow Assistant|
|--|--|--|
|Sync the google sheet of new employee joinees with device register sheet from IT|🤷|✅|
|Export the client leads from Apollo for the financial compliance usecase into Salesforce|❌|✅|

### Agent Architecture
<img src="https://hackmd.io/_uploads/Bk2ymWDaA.png" height="670px"><br>
Here you can use anything for the orchestration engine and trigger management in the open source space. Obviously Zapier does not have an API exposed but a lot of other players like n8n and all do have an API that the model can follow for workflow schema.

> **Quick Hack:** If your core platform does not have a legible API, you can use an internal tool builder to support workflow creation and then just embed it into your webapp.

## Level 4: User Journey Manager
Now we are getting into the really "generative" part of SAAS 2.0. These are not multi-agent systems but certainly do pack in some of their complexities.

> Note: Donot implement any of this unless you have the other 3 sorted out perfectly and have a great GraphQL API setup with very legible resolvers.

The overall premise of this is that not all your customers have been there from v1. And they have not seen the product evolve over time into the feature dense offering you have today. This makes it hard for them to get value out of the product all at once.
\
Since every page on your UI is only some view on an excel table, you can continuously analyze usage data to identify journeys with low completion rates. Once you have identified which steps of the journey need to be simplified, you can generate multiple alternate subviews.
\
You can qualify these subviews based on user persona and that particular person's business context. Once you pick any one subview, you can pass it onto the UI engine as well as your product walkthrough builder to help the user get a better grasp of things.

### Agent Architecture
<img src="https://hackmd.io/_uploads/rJZGD7wpA.png" height="870px"><br>

> Note: In this agent specifically, there are a lot of scenarios that you need to handle around field selection because not all of them are independent fields. Some are derivative, some are redundant in the new UI anyway and some fields might be needed in combination to render a single UI component.

A final point of understanding is your user experience will not look the same when a few fields have simply been deleted, this may affect layouts and needs to be handled gracefully on the frontend. Else you will end up with a patchy UI.

## Level 5: E-R Innovator
Horizontal SAAS products often miss out on nuances of different sectors or business size and stage. This leaves them open to disruption even at very mature stages. At some point they will need an AI agent that can reliably and "contextually" fork their data model across business contexts.
\
A core component would be look at your usage data, identify relevant counter-patterns in product usage and combine it with the user's a business context. This information can be used by the LLM to identify key changes in the schema that can be done to fit user behavior better.
\
The goal of this agent is slightly different from **User Journey Manager Agent** as it tries to adapt the data model itself to support additional requirements and fields without needing a layer of "product management".

### Agent Architecture
<img src="https://hackmd.io/_uploads/HyXdcxwp0.png" height="720px" style="margin-bottom:50px;">

> Note: Donot try to implement this unless you are sure you have a very modular and legible data model and your frontend will be able to support any non-trivial migrations without spoiling UX.

<br>

>Note 2: Only implement this when you are perfectly sure that all your other agents are working reliably and you have seen consistent improvement.

<br>

## Conclusion
Once you have done these things, you have future proofed your product against the kind of replacement that is coming for most systems of record. Well atleast until the next generation of tools like [LLaMaCoder](https://llamacoder.together.ai/) are not viable 😂😂.