Top 10 tech liabilities in Ucommerce
===
---
### _Fixing the Core_
---
### _Fixing the Core of the product_
---
### Fixing the core of the core
* process
* product internals
* tools
* mindsets
---
### 100% my personal perspective
Your mileage may vary!
---
### A product as a business
* Assets = everything that makes you money or can make you money in the future
* Liabilities = everything that costs money or will in time
---
Fortunately, the assets are far greater than the liabilities
---
Any business needs to constantly trim liabilities
---
Before we can fix liabilities
we need to identify them
---
I'm a pessimist
which turns out to be a valuable trait when looking for things to improve
---
## Liabilities
* QA proces is mostly manual & labour intensive
* Fixed release dates hamper quality
* XML config files are time sinks
* Confusion about version compatibility
* Development speed on the backend is catastrophically slow
* Database-in-database, over-normalized db design
* NuGet itself is in flux
* Our data layer is a mashup of patterns
* Virtualization, so far, is for the few
* Ucommerce is tailored to 3 categories and 9 products
---
## Liability: QA proces is mostly manual & labour intensive
Our test coverage is mediocre.
Most sub systems are _legacy_, including much of the new stuff.
Current Team City setup is not isolated nor scalable
---
Goal: Improve our ability to move fast with _confidence_.
---
A similar product I worked on recently had 1/3 of all developers be DevOps (5 people).
Only job: Trim the production line.
---
#### Actions:
1. Invest in writing cypress and other kinds of tests for the Catalog and Products apps.
1. Hire or re-assign a dedicated DevOps person.
1. Virtualize TeamCity agents so that a new, fresh agent (with all deps installed) is spun up per build.
---
### Liability: Fixed release dates hamper quality
Everyone runs as fast as they can during the last month.
Quick and dirty decisions are made.
Quality suffers.
More and more last minute decisions get baked into the product.
---
#### Actions:
1. Move to another release plan:
1. Continuously
1. When product is ready
1. Actual Beta period. With real beta testers.
---
### Liability: XML config files are time sinks
They get out of sync
Many ways to make mistakes
Only support a subset of IoC features
---
Makes it 3x harder to refactor code.
It's not safe by default.
---
#### Actions:
1. Registration through code
1. Move to a modern IoC
(more on this next Thursday)
---
## Liability: Confusion about compatibility requirements
During development and QA
---
What does Ucommerce support?
__.NET__ 4.6? 4.7? 4.7.2? 4.8?
__MSSQL__ 2019? 2017? 2016? Earlier?
__Visual Studio__ 2019? 2017? Earlier?
Chome? Firefox? Safari? Edge?
__Sitecore__ 8? 9.0? 9.1? 9.3?
---
Which upgrade scenarios do we offer?
That's another 3D matrix right there.
---
#### Actions:
1. Make the compatibility table part of the public roadmap
2. Automate testing all combinations of supported dimensions (only feasible with virtualization)
---
## Liability: Development speed on the backend is catastrophically slow
* The competition do not wait for us
* Trains of thought derail every day
---
### Build speed
Typical **frontend** development cycle:
| | |
|:---|---:|
| think | 1 min |
| code change | 1 min |
| _compile + hot reload_ | 5 sec |
| _run to breakpoint (browser debugger)_ | 15 sec |
| TOTAL | **2,33 min** |
---
#### Frontend
| | | |
|:---|---:|---:|
| Cycles per day | 7 * 60 / 2,33 | **180** |
| Total wait time per day per developer | 20 sec * 180 | **1 hour** |
---
### Build speed
| | |
|:---|---:|
| think | 1 min |
| code change | 1 min |
| _compile_ | 2 min |
| _site restarts_ | 1 min |
| _navigate to the menu item_ | 30 sec |
| _attach debugger_ | 30 sec |
| _run to breakpoint_ | 15 sec |
| TOTAL | **6,25 min** |
---
#### Backend
| | | |
|:---|---:|---:|
| Cycles per day | 7 * 60 / 6,25 | **67** |
| Total wait time per day per developer | 4,25 min * 67 | **4,75 hours** |
---
### Goals
Deliver more features faster
Feel less pain
Continously optimize flow
---
#### Actions:
1. Build into one folder to speed up raw build time (more on that next Thursday!)
1. Decouple Ucommerce from CMS's for every day development
1. Have all the OSS repos simultaneously be part of the main SLN (with project references) _and_ standalone (with NuGét references.)
1. Investigate how moving to .NET Core can possitively affect development cycle
(more on this next Thursday)
---
## Liability: Over-normalized db design
---
Here's a database design I made in 2002 for a CMS I built...
---
```graphviz
digraph hierarchy {
nodesep=1.0 // increases the separation between nodes
node [color=Gray,fontname=Courier,shape=box] //All nodes will this shape and colour
edge [color=Gray, style=solid] //All the lines look like this
DocumentType
Document
Property
Field
Field->DocumentType
Property->Field
Document->DocumentType
Property->Document
}
```
---
One year, failing to provide decent performance, I promised myself I would never ever again build a "database in a database".
---
1. Slow queries beyond optimization in the backoffice.
2. Hard to reason about code. Slow learning curve. Ripping of hair.
---
#### Actions:
1. Denormalize the database for content (leave definition sub system as is.)
---
## Liability: NuGet itself is in flux
---
`install.ps1` isn't exactly deprecated, but it's a feature unique to packages.config and requires installation through Visual Studio.
MS is modernizing NuGet and csproj file formats for .NET Core.
Eventually, everyone will be adapting _PackageReference_.
---
Goal: Install ucommerce from powershell to make it automated and testable.
---
#### Actions:
1. Move to PackageReference format.
(Instead of `install.ps1`, use a custom MSBuild target to install Ucommerce.)
---
## Liability: Our data layer is a mashup of patterns.
Three patterns used:
* Active Record
* Unit of Work
* Repository
---
All are good patterns.
Individually.
Together they are a disaster.
---
Main issue:
Most calls to a repository can have side effects anywhere in the database.
Impossible to reason about code.
---
NHibernate itself mainly implements the UnitOfWork pattern (the session).
But it's old, bloated and full of complexity.
If you don't believe me, please explain how caching works in NH.
---
#### Actions:
1. Replace NHibernate with Dapper, a much simpler and less intrusive API.
---
## Liability: Virtualization, so far, is for the few
---
Benefits of virtualization:
* Isolate complete runtime environments to make them predictable
* Rewind entire server to a point in time
* Quickly test various versions of prerequisites, CMS's and Ucommerce
---
#### Actions:
1. Finish making VM infrastructure available and accessible on every type of machine.
---
## Liability: Ucommerce is tailored to 3 categories and 9 products
And performance suffers for it.
We have not yet reaped the benefits of seeded databases.
---
#### Actions:
1. Ship Ucommerce with the seeded database and an option to quickly delete the seeded data
1. Produce reset targets with pre-seeded databases.
1. Change the Cypress tests to run on seeded database.
---
## Liabilities
* QA proces is mostly manual & labour intensive
* Fixed release dates hamper quality
* XML config files are time sinks
* Confusion about version compatibility
* Development speed on the backend is catastrophically slow
* Database-in-database, over-normalized db design
* NuGet itself is in flux
* Our data layer is a mashup of patterns
* Virtualization, so far, is for the few
* Ucommerce is tailored to 3 categories and 9 products
---
## All actions
1. Virtualize TeamCity agents so that a new, fresh agent (with all deps installed) is spun up per build.
1. Invest in writing cypress and other kinds of tests for the Catalog and Products apps.
1. Hire or re-assign a dedicated DevOps person.
1. Move to another release plan
1. Actual Beta period. With real beta testers.
1. Registration through code
1. Move to a modern IoC
1. Make the compatibility table part of the public roadmap
2. Automate testing all combinations of supported dimensions (only feasible with virtualization)
1. Build into one folder to speed up raw build time (more on that next Thursday!)
1. Decouple Ucommerce from CMS's for every day development
1. Have all the OSS repos simultaneously be part of the main SLN (with project references) _and_ standalone (with NuGét references.)
1. Investigate how moving to .NET Core can possitively affect development cycle
1. Denormalize the database for content (leave definition sub system as is.)
1. Move to PackageReference format.
1. Replace NHibernate with Dapper, a much simpler and less intrusive API.
1. Finish making VM infrastructure available and accessible on every type of machine.
1. Ship Ucommerce with the seeded database and an option to quickly delete the seeded data
1. Produce reset targets with pre-seeded databases.
1. Change the Cypress tests to run on seeded database.
---
### All actions
No time for them all.
Invest in assets.
Invest in reducing liabilities.
---
### How to decide what to improve?
Economic decision
Cost of delay vs. cost of doing
---
### Cost of delay
Make visible => data
---
### Cost of doing
Experiment => better predictions
---
#### Next Thursday
## Reducing Solution Complexity
{"metaMigratedAt":"2023-06-14T22:25:24.770Z","metaMigratedFrom":"YAML","title":"Top 10 tech liabilities in Ucommerce","breaks":true,"contributors":"[{\"id\":\"7e34ea49-c725-4e89-bb55-e9383bc476f9\",\"add\":29121,\"del\":19316}]"}