---
tags: OpenEMIT
---
# {OpenEMIT|Design}
>[name=George Willis, Founder of OpenLedger]

{%hackmd p6TgTK-8SlKQzxLCsSEH9g %}

**What is a distributed ledger (blockchain) vs a distributed log?** If they both store entries into a *journal of record* in an "append-only, no eraser" fashion, ***just like an accounting ledger***, what is the difference?

If **Serverless** (see [CNCF Serverless Landscape](https://landscape.cncf.io/serverless)) and **Function-as-a-Service (FaaS -- IBM Cloud Functions, AWS Lambda, Google Cloud Functions, Azure Functions, etc.)** can **++react++** to data streams, **isn't all of the above "Data-Driven" or "Event-Driven"?**

If a ***distributed Key/Value (KV) store*** like AWS S3, Apple FoundtionDB, Snowflake, Couchbase, Yugabyte, Cassandra, etc) (or any distributed SQL database for that matter,) is used to store ledger entries, and ***triggers on insert used to "react and invoke processing",* doesn't this yield the same mechanism?**

### In all above cases, we can add entries to a "ledger", which trigger processing with the entries as input. It's all data-driven ***"reactive"*** processing, right?

If so, could an ***Open/Interoperable Framework*** be created with "plug-n-play" adapters for various **"ledger technology"** one wishes to leverage, so that ++capabilities++ like **++storage, distribution, and "reactive invocation++" of "entries" could all be handled by configuration of these technologies, and developers could just code to this standard API/Framework? **This would allow switching providers of these capabilities for any reason -- cost, performance, consolidation, evaluation, etc. -- ++without recoding!++**

> First, some teminology. Let's term the "append-only, no erasure" concept ***immutability*** or "unchangeable". Let's term an ordered, immutable "journal of record" a ***ledger*** -- just like in accounting, ***because that is the capability sought -- accountability*** (non-reputability and authenticity in security lingo.)
>
> With this in mind, the difference between distributed logs and blockchain is how they implement ***immutability***. In blockchain, the accountibility sought is amongst a community of unknown, untrusted membership, so strong encryption is used. (In this context, "membership" means those with access to the ledger.)
>
> In distributed logs (Kafka, Redpanda, Pulsar, etc.) you typically operate inside secure "zones" which have more trust in the membership (access) within the zone. If needed, encryption can be employed through various mechanisms to increase trust in the logs authenticity.
>
> In fact, one could technically think of a distributed ledger as a distributed log with strong encryption (known as proofs).
>
> ### ***The similarities are everthing else*** -- and that is the key takeaway. They are both "ledgers".
>
> They both ***publish*** entries. They both have ***subscribers*** that receive the entries. They both are ***immutable by design***, and ***how they achieve immutability is not an application concern --- ++it's an infrastructure concern++.***
>
> ***Serverless*** and ***FaaS*** are primary used for "data-driven" applications, and can easily use ledger entries to "react".
>
> In software architecture terminology, **Pub/Sub or "reactive invocation" is known as *implicit invocation***, in contrast to ***explicit invocation*** used in REST. (see [Hooks over Arrows](https://)).
>
> ### Pub/Sub invocation yields asynchonous, loose-coupling of invocation.
> * **Naturally Asynchronous** -- no distributed synchronous call stacks to unwind on call failure, or "idle" (blocked) execution
> * **Natural Parallelism** across cores and clusters due to multi-subscriber capability
> * **Cloud Serverless** Scaling and High Availability (HA) (Container pools of redundant subscribers)
> * **CQRS** capabilities
>
> ***"Command/Query Responsibility Separation" (CQRS)*** separates ledger "Commands" written to the ledger, from Database "Projections" used for query operations. This provides ***independent scaling of reads vs writes***. Moreover, it allows ***multiple "projections" of the same ledger data for query optimization via the underlying datamodel of the database used.*** Some data queries are optimized by relational models, others by graph, network, hierarchical, columnar, object, etc. CQRS can not only scale the number of query services, it enables ***[Polyglot Persistence](https://hackolade.com/polyglot-data-modeling.html).***
>
> Architectures that are driven by entries know as "events" are termed ***Event-Driven Architectures.*** We wholeheartedly recommend ledger entries be ***events***, as it is a clarifying concept of ***reactive shared messaging*** (as opposed to shared data resources.), and are associated with further capabilities known as **"Event Sourcing"**.
>
> #### Note: the entry format is ++open++. Events are supported as one of many possible entry formats, but developers are free to choose any format. Data-driven applications that use "events" as entries are Event-Driven. ++Version controlled schemas frameworks++ like Apache Avro and Parquet are prime examples of popular tools to manage your entry formats.
>
> Stream and batch processing are both configurable in this [Kappa Architecture](https://hazelcast.com/glossary/kappa-architecture/). ***Evolutionary Development*** allows new workflows to be derived by branching from existing workflows ***without impact to existing workflows.*** This is a byproduct of ***parallelism.***
>
> Lastly, one might ask why we highlight that **OpenEMIT** is ***"Coded in Rust"***. Rust offers strong typing, safe memory and resource management, "fearless concurrency" -- all without untimely garbage collection interrupts. When security, correctness, lean resource management and performance are paramount, Rust is a superb choice.
> [color=blue]

## In summation
These are the concepts that enable ***OpenEMIT***. The value of open architecture, interoperability, cost control, evolutionary infrastructure, evolutionary development, simplicity, scaling, availability, and development velocity **are so numerous -- they are overwhelming to describe and qualify.** Please accept this humble attempt to do so.