# Emergent Trends in Modern Applications :chart_with_upwards_trend:
## Thinking In Bets :moneybag:

:man: [Rohit Kelapure](https://www.linkedin.com/in/rohitkelapure) :calendar: February, 2019 :iphone: [@rkela](https://twitter.com/RKela)

---

# ==1. Visual Programming Takes Hold==

###### tags: `low code`

New low-code or no-code ventures are making it easier and faster for teams to build software tools, websites or apps. [wsj-paywall-article](https://www.wsj.com/articles/some-startups-are-taking-the-coding-out-of-software-development-11571396400). There has been a proliferation of low-code and no-code systems in the past 5 years.


## WHY
- Citizen programmers :computer:
- Developers are the new Kingmakers
- Programmer demand outstripping supply
- Democratization of the Code
- Designers need to code too :grin:
- AI/ML patterns can be applied to application development
- Finally these low code systems have crossed the developer chasm and are good enough for developers to build, extend and write complex programs
- Massive Expansion Opportunity : 800M+ knowledge workers, 100M+ Technical Team Members, 23M+ Software Developers

## Validation
* WebFlow - *Better for Coders*
* AirTable - *Excel on Steroids/millenials*
* Bubble - *Better for Designers*
* Unqork - *Low Code Systems for Enterprises - Footprint in financial domain*
* Outsystems
* Retool - *Intranet apps*
* Mendix ^acquired^
* AppSheets ^acquired^
* Tableau

# Contrarian But Right Bet
Build the Kubernetes/Spring of visual programming - an Open Source Java/Javascript based Visual Programming frontend and backend Runtime Environment to foster an open ecosystem of plugins for visual programming. Monetize the open extension points and enterprise integrations into backend systems. This is the next step in the evolution from static to dynamic website generators.

---

# ==2. API Gateways are Back !==

###### tags: `API gateways`

There is much confusion in the API world due to the overlap of feature and function between (east-west) service meshes that provide telemetry, security and service discovery and (north-south) API gateways that provide security, throttling, governance, discoverability and a central point for ingress. Furthermore API gateway vendors and Service Mesh vendors portray their products as ones swing on both sides :baseball: .

^https://a16z.com/2019/10/04/commercializing-open-source/^`

## WHY
- Old school SOA is back - vendors have become hippier
- Total Confusion in this space
- All Problems are Integration Problems [2020-the-year-of-integration](https://sdtimes.com/data/2020-the-year-of-integration/)
- ITIL driven BUY first model and stitch it together
- Developer first LENS for the strategic Business services
- Enterprise API Fabric for developer services

## Product Validation
- Mulesoft
- Apigee
- Envoy/DIY
- Istio/Linkerd
- Kong
- Apollo GraphQL server

# Contrarian But Right Bet
While the big dogs (Google, Mulesoft) in this space duke it out, there is an interesting bet to be placed here in the very tight integration of [Spring Cloud Gateway](https://pivotal.io/platform/services-marketplace/application-tier-networking-and-gateway/spring-cloud-gateway) and other developer first gateways with with [Apollo GraphQL Server](https://www.apollographql.com/docs/apollo-server/) and doubling down the developer declarative DSL of the API Gateway. See [apollo-graphql](https://a16z.com/2019/06/12/apollo-graphql/). For this bet to work the acquisition flow has to be through developers with zero sales touch at a low price point and with remarkable quality. This is a bottoms-up play in the first two years.

---

# ==3. Monoliths to Microservices & The Rise Of Serverless==

###### tags: `mono2micro` `mono2nano`

Consultants and SIs big & small are making tons of money migrating and modernizing legacy apps to the cloud. This is a wave that is lifting all boats. With roughly `10%-15%` of all workloads currently running the cloud. There is significant potential here in capturing leverage with tooling to aid and accelerate modernization to the cloud. Since services are typically 10x product spend - I believe there is significant money to be made on the tooling side for helping enteprises ramp into the cloud.

Datadog state of the Serverless [survey](https://www.datadoghq.com/state-of-serverless/?ref=announcement-banner) clearly indicates that Serverless is no longer a buzz word. Forward looking developers that flocked to microservices and containers are now moving up the stack to lambda and functions as evidenced by the fact that half of AWS users have adopted lambda. The same trend can be extrapolated to ther public clouds. Per the survey nearly 80 percent of organizations in AWS that are running containers have adopted Lambda.


To break this down - what are the top challenge associated with migration from monoliths to microservices ?

:bomb:

1. Database migration.
1. Divide business capabilities in sub-business capabilities.
1. Candidate microservices still need expert judgment before being developed in practice.
1. Distributed system development needs skilled developers.
1. Resources management.
1. Complex environment setting, many automation tools are needed.
1. Transactions are notoriously difficult to implement.
1. The organization must adapt to the new way of working.

[*Ponce, Francisco & Márquez, Gastón & Astudillo, Hernán. (2019). Migrating from monolithic architecture to microservices: A Rapid Review.*](https://www.researchgate.net/publication/335716451_Migrating_from_monolithic_architecture_to_microservices_A_Rapid_Review)

# Bets
1. Machine Learning to automate the decomposition of monoliths into microservices and functions, combining Domain Driven Design principles, dynamic runtime flow analysis of code (provided by an APM agent) with static code analysis patterns. [vFunction](https://www.vfunction.com/#the-landscape)
2. SaaS for Cloud Native Readiness portfolio assessment, reporting and code remediation with full devops lifecycle visibility into application migration & modernization.
3. Automation, Runbooks and templating _like_ [Bitnami Stacksmith](https://bitnami.com/stacksmith), and [Replicated](https://www.replicated.com/) for bootification & containerization of legacy workloads using cloud native frameworks and tech like spring boot to microservices and functions.
4. Vm2Container solution that automates containerization of stateful and stateless workloads from VM to containers. _see_ [google anthos](https://cloud.google.com/migrate/anthos) and [anthos-headache](https://docs.google.com/document/d/1lF1a46G8fixhs-db5ETCIlWFLgkReORDBHv6PmwnmuI/edit?usp=sharing)
5. Integration of an Open FaaS runtime with K8s platform and backing services is critical otherwise AWS/Azure/GCP will create proprietary lockin. The lack of consolidation and fragementation in FaaS & serverless frameworks has hurt open platform plays and opened a desirable window for hyperscale vendors.
6. Zero friction, low touch onboarding of apps onto the platform. Create higher abstraction Spring/Java microservices/serverless building blocks for developers. Significantly improve the UX and only bring in sales/consulting for high end customer segments. Get the flywheel of the framework ecosystem revving with partner channels.

###### `Atlassian Playbook` ^https://s2.q4cdn.com/141359120/files/doc_presentations/Investor-Session-2019_041019.pdf^

---

# ==4. Opportunities in Inner Loop development with Containers==

###### tags: `inner-loop developer workflow`

The Inner loop of backed containerized service development is under flux. A typical deployment now consists of a mixture of hybrid on-prem and off-prem services and containers running the cloud. The developer workflow for inner loop development is still up for grabs. The Rails Moment for k8s dev is yet to arrive.

## WHY
- Patterns of enterprise dev are segmented *(credit @r_arni)*
1. Low Code
- visual programmer/ designer
3. git push - cf dev guy
- KNative
- kf
4. **Container control** - Docker guy
- NEW DOCKER
- Cloud run
5. **K8s access - inner loop dev flow**
- Pivotal Build Service for pre-push => build service
- Skaffold for Templating++
6. Full power of K8s
- kubectl
- What is the % Time spent in the inner & outer loop.

## Validation
- Google cloud code
- Skaffold into VSCode and IntelliJ
- transparent push to GKE
- skips CI/CD
- VMWare Fusion embeds K8s - can run containers, VMs, and k8S manifests

## Bet
Its critical to fully build out the circle of code story with close integration with CI and CD tools. Increasingly CI tools will be aggregated and provided by the source code vendors themselves. Git vendors like Gitlab, Microsoft and Atlassian and IDE vendors like IntelliJ will own the market for CI tooling. CI is shifting left closer to the source similar to security scanning `#DevSecOps`.

Jenkins and Bamboo are still the leaders in the market. Not placing this bet will result in a value capture risk. If the circle of code is not built close to the target platform there is real risk of getting disintermediated at source by the CI/Source code vendor of choice.

An interesting bet is here is the deeper integration of CI Tools (Concourse/CircleCI) with Mattermost as a collaborative open source slack clone using it to drive the central nervous system of devops. As a cautionary note [atomist](https://atomist.com/) has still not reached product market fit.

---

# ==5. From Imperative to Declarative Systems==

###### tags: `declarative systems`

## WHY

- Reduction in the complexity of state
- Cognitive Overload
- System optimization rather than local maximas
- `gitops` style declarative configuration as data model is taking off in CI/CD Tools and other forms of configuration and operations management. [GitOps](https://www.weave.works/technologies/gitops/) style model uses operator k8s native reconciliation loops in the cluster to trigger deployments inside Kubernetes, which means that you don’t need a separate continuous delivery tool

> GitOps is the new DevOps

## Validation
- Imperative => Reactive Data Pipelines (Ascend)
- Chef, Ansbile => Terraform, Cluster-API
- Spring Web MVC=> Spring Reactive
- Jenkins => ArgoFlux - gitops

## Bet

With both data and compute moving to declarative paradigms the current bet on reactive streaming based systems seems to be working. The UX of declarative systems however leaves much to be desired and whoever cracks the code of getting developers efficient & enabled with reactive and declarative systems could be the next Spring Boot/.NET Core.

---

# ==6. Spring Boot Competition==

###### tags: `server-side web frameworks`
`https://www.jrebel.com/blog/2020-java-technology-report`

## WHY
### Perceptions ?
- Fat - *High Memory Footprint*
- Slow - *Startup Time*
- Too much Magic - *Peeling back the onion is tough*
- Anyone BUT Spring movement is consolidating under
- Quarkus
- Micronaut
- Microprofile
- Jakarta EE

## Validation
- Control Points (JIRA) on CD
- Winner take all - 1..3 solutions / multiplicity
- Network effects - winner take all
- Built in network - distribution effect
- Network effects - control points
- **Adjacencies, Adjacencies, Adjacencies**
- JFrog - #1
- Gitlab vs Jenkins (hand crafted)

## Bet
These bets have already been placed
1. Accelerate Spring Framwork/Spring Boot GA adoption of GraalVM.
2. Invest in optimization of Spring Boot memory footprint, startuptime super-optimized docker container image generation.
4. Low sales touch, high quality, developer driven acquisition.

---

# ==7. Streaming Data Platforms==

###### tags: `data pipelines`

Hadoop Data Lakes are turning into S3 based cloud data lakes drained by data meshes & data pipelines catalyzed by the emergence of streaming frameworks like kafka & Spark. These data lakes are giving way to a federated purpose built pipelines for different data types. The separation of compute and storage layer continues in the data analytics world as workloads shift to the cloud. Open-source column-oriented data storage formats like Apache Parquet are standardizing persistence leading to better interop between between tools.


^https://martinfowler.com/articles/data-monolith-to-mesh.html^

## Product Validation
- Confluent Kafka _all streaming workloads_
- Apache Spark _more for machine learning_
- Airflow _In memory fast query processing big data_
- Presto _SQL Based Query Fabric for Hadoop_
- Ascend _Swiss Knife of data pipelines_

## Bets
- @KnativeProject + @apachekafka/Pulsar/RMQ could become the substrate of the next set of platforms built on top of @kubernetes.
- RMQ becomes the message broker of choice in cloud native workloads. Continued investments in Postgres as the relational datastore of choice.
- Data domain transformation and migration necessitates high touch consulting services to help enterprises break the chokehold of legacy data.
- Confluent Kafka will emerge as the streaming tech of choice. Married with the power of spring developer abstractions eventing will become front and center in a loosely couple event driven distributed world.

---

# ==8. Hyper-specialized Platforms on top of Kubernetes==

###### tags: `kubernetes distros`

1. @kubernetesio will eventually disappear. @kubernetesio will be consumed like linux distros. Hyper specialized PaaSs like Render, K3s, etc., that specialize @kubernetesio for edge workloads or verticals like Retail will arrive. This will be a key part of the disappearance of @kubernetesio
3. The container scheduler is just one of the manifestations of the API model - core runtime of @kubernetesio . #Kubernetes is NOT just for container schedulers. Entire platforms and vertical products can be built on it
4. @kubernetesio is rapidly iterating to its Rails moment. Nothing authoritative has arrived yet.
6. Just like you need a licensed plumber to fix a water main leak - you will need a certified kubernetes administrators to go under hood when the @kubernetesio platform breaks. Don't be scared of @kubernetesio sys admins and developers simply have to do a mapping exercise to remap their mental models with the best practices of @kubernetesio
9. Control planes for clusters have to think out of the box. @kubernetesio Cluster management is not a solved problem

## WHY
- Industry specific PaaS's and platforms (Render)
- Developer workflow enhancements like kNative
- Kubernetes for edge distributions
- Resurgence of DIY with K8s

## Validation
- Edge specific distros like K3s
- Render
- Rancher for better UI- Cluster Management

## Bets
- Pay attention to niche, custom and vertical K8s distributions that are satisfying unmet needs like edge deployment. Monetize platforms on top of Kubernetes.
- Double down on developer experience with Cloud Foundry. The Heroku `git push` and `cf push` experience has survived ten years and is really productive to developers.

---
# ==9. Observability (`#O11Y`)==

###### tags: `observability`

The average enterprise application has pretty poor hygiene when it comes to monitoring and metrics. Furthermore this problem is getting worse due to the plethora of deployment and cloud options available. Most app developers seem to over-rotate towards spunk for logs and metrics.

Its fun to see old school APM vendors (NewRelic, AppDynamics, Dynatrace), Logging vendors (Splunk) and new age `#O11Y` vendors duke it out in an all out battle of determining root cause and understanding the visibility of systems. Some of the vendors also venture into the continous delivery space as rollbacks of deployments can now be done based on performance profiles or SLO definitions. Every vendor is acquiring a combination of `{logs, metrics, traces}` products and is in a mad rush to integrate them to provide a holistic window into blackbox systems to their customers.

There is an interesting collision here between the old and the new when it comes to visbility into deep systems. The market is consolidating as companies realize that an integrated model of all three i.e. logging, metrics, traces and profiling information at full fidelity is needed to determine what went wrong when a 1 in million event occurs in distrubuted systems.

## WHY
- Management of runtime state for complex distributed systems
- Emergence of high cardinality and high dimensionality systems
- AI/ML based interrogation of logs and metrics
- Deep focus on distributed traces and logs

## Validation
- Honeycomb - *Leading SRE and `#O11Y thoughtleadership`*
- Lightstep - *Deploy with confidence*
- Datadog - *Does full stack monitoring*
- NewRelic - *Marketplace program is promising*
- AppDynamics - *Excellent Sales force - F500 penetration*
- Wavefront - *Cloud SaaS VMware Play*

## Bets
My money here is on a dark horse i.e. Honeycomb which under the leadership of CTO Charity Majors has defined this new field of `O11Y` and will continue to lead it. The potential TAM of this space is only bound by the size of the microservices/serverless/hybrid universe.

---