Getting started with Frontend First at Muze

# Getting started with Frontend First at Muze Muze is a web development agency, founded in 1998. We love the web, and we think it can be better. So we're building tools and software to help improve the web. One of the things we want to improve is to allow more people to build on the web. Today this means you need to be a (web) developer. But not everyone is cut out for that. Instead of turning more people into developers, we'd like to turn the web into a simpler, easier system to make stuff for. Previously we've tried to do this by building a Content Management System (CMS), called Ariadne. You can still get it at https://ariadne-cms.org/. It works, its technically sound, but it isn't easy to develop in. And the rest of the world has decided to use Wordpress for just about anything. One of the main lessons we learned from building Ariadne, and trying to get people invested in it, is that people don't like to switch to systems that require a completely new way of working. One where they can't use their existing knowledge and tools they are used to. So we're now trying a new approach. This time it is not about building a complete system, but instead a set of components. Each component works seperately from the rest, but they all work well together. The components aren't a framework, so they can work with whatever system, framework or architecture you like. You can mix and match them with existing components, libraries, etc. And you can use all your own preferred development tools. ## So what are the components and why do we have those? From a high level perspective the components are: 1 - [A component design tool - SimplyDesign](http://simplydesign.dev.muze.nl/design-system/). It is focused on writing CSS, but in a way that makes it easy to create a design system and/or component library. 2 - [A page builder - as yet unnamed](https://designblocks.simplyedit.io/design.html) (and unfinished). Here you can drag and drop components to create pages - or parts of pages. In effect it creates just HTML fragments. 3 - A form builder - also unnamed (and unfinished). This is just a more specialized version of the page builder. 4 - [A HTML (or anything) editor - SimplyEdit](https://simplyedit.io). This is actually more than just an editor, it is also a web render system, similar to Vue or Angular. 5 - [A webapp builder - SimplyView](https://github.com/simplyedit/simplyview). This is a set of programming components that make it easier to build a frontend application. 6 - App and Site Hosting - unnamed. Based on an Apache and PHP combination, this makes it easy to host sites and applications. Currently there is a very basic implementation called [SimplyEdit Backend](https://github.com/simplyedit/simply/simply-edit-backend). 7 - [API components, e.g. data and file storage - ARC](https://github.com/ariadne-cms/arc-arc). This is a set of PHP components you can combine for a number of different storage needs. From a straightforward REST API (well, JSON-over-HTTP api) to an Ariadne-like CMS. [A set of prototype API's](https://github.com/poef/arc-rest) are in development in my personal Github space. Each of these can be used standalone. And each uses standard formats (HTML, CSS, JSON) to exchange information with other components. ## Why this seperation? In our experience when you have an idea of a software project, most people think about the frontend first. Developers have learned instead to think about the data model and scalability and architecture first. This results in lots of work by developers that is only really evaluated when the first frontend code is running. Any failures in design or miscommunication is only found at that point. So a lot of work may have to be thrown away. By focusing on building a working frontend as soon as possible, we minimize the loss of time and work. The end users get a prototype they can really try and use which gives us feedback on whether the whole idea is valid and if the approach is valid. The main point is to create the shortest possible feedback cycles, with high quality feedback by actual users. So instead of defining data models, we build a 'view' in HTML. This view defines (part of) the data model. We store the data as defined in the view. There is no security or validation in the first prototype. We go for a high quality, low effort look and feel, with a low quality and low effort backend. This is done by using a standardized design system and component library, which is of high quality. This design system is build to be easy to style and adjust to any branding or design requirements. In our experience, if you make a prototype that looks unfinished and undesigned, users will not take it seriously and will not use it seriously. The quality of the feedback will suffer. Applications and websites are all unique. But you can get away with using standard components as long as you allow custom styling. Some applications and websites will need custom components, so the component design tool should allow customization and extension. Pages and forms should only contain predefined components. If you need something custom, this should be build in the component design tool, not the page builder. However, each page may need custom code around the components. So the pagebuilder consists of selecting (or writing) a page framework and then drag/dropping the needed components in place. The page framework may contain multiple page designs or templates and a way to switch between them. This allows you to make a Single Page Application (SPA). An extension of this is a Progressive Web App (PWA). There are cookbook examples of many of these, as well as normal web sites with multiple page templates / frameworks. When the design is finished, we publish the HTML, CSS and JavaScript on a hosting server. The application or website framework defines the folder structure. We standardize on Apache and PHP for server side logic, using SimplyEdit and the SimplyEdit Backend as the default for page rendering and editing (CMS). This means that page content is rendered in the browser, by SimplyEdit and sent over HTTP as JSON. If the browser doesn't support javascript, the site won't work. Where this is a problem, we use tools to run the javascript in a server proxy that mimics a browser. Usually we will have a development hosting environment and a live environment. Changes in code (HTML, CSS, JavaScript) or made on the development environment and committed to git. The live environment is then updated from git. In the future we will also create staging environments where changes in website or application content can be staged. Finally applications usually require access to API's or more general - server side logic and storage. For this we've build our own set of server side components under the ARC umbrella. These contain abstractions that we've extracted from 20+ years of experience in building applications on top of Ariadne, our home grown CMS. But ARC components are true components. You can use them seperately or in combination, and they don't force specific application architectures. They do however bring to the table a set of abstractions that are a bit unique, compared to most other PHP libraries. The focus is on building web API's, ofcourse. But it doesn't contain well-known paradigms like Model-View-Controller (MVC), API Routing and ORM's (Object Relational Mapping). ARC instead focuses on Path and Tree abstractions. These make it easy to map incoming URL's to a tree based Store. Data types in this tree store make content-based or data-based routing possible. ## Why build our own? In short, nobody else is building these tools the way we'd like. There are plenty of libraries and components build by others that we do like and use. But these are all special in some ways. So let's start: ### SimplyDesign. There are other tools to build design systems, the most well known is probably [StorybookJS](https://storybook.js.org/. But it includes a lot of stuff we don't need and/or don't like. Things we like: - instant (visual) feedback - CSS Custom Properties Things we hate: - unnecessary abstractions (React) - compilers for CSS (SCSS) - code in HTML (Angular, Vue) - non-standard formats (all of the above) Muze builds components as basic HTML and CSS combinations, with maybe a tiny bit of javascript. The idea is that anyone with a good understanding of these should be able to use and adapt the components. We don't want extra compilation steps and extra languages to learn. The design system (ds-) we've built uses only standard browser supported CSS. It is based on principles outlayed in [Atromic Design](https://atomicdesign.bradfrost.com/) by Brad Frost, as well as [ITCSS](https://www.xfive.co/blog/itcss-scalable-maintainable-css-architecture/). Most importantly, it uses a prefix for all classes, so no classname conflicts will arise (looking at you Bootstrap). Whenever a component needs custom behaviour, that behaviour is written in javascript and triggered using custom data attributes, which are also namespaced. These attributes use general components from SimplyView. ### Page builder / Form builder There are open source page design tools available, but they either try to do everyting visually ([GrapesJS](https://grapesjs.org/)) or are based on a specific CSS design system (BootstrapUI, TailwindUI). These solutions are either too tied to things we don't like (bootstrap, tailwind) or try to do too much in a single component, making it too complex. The page builder we envision has very few features. This also makes it fairly simple to build. For now we mostly build a page framework by hand, as code and add the needed components by hand as well. The form builder is probably more important later, as it is also a way to define the data format you want to use and probably store. This needs some more thought. There is a prototype that has actually been used in production. Some lessons learned there need to be implemented in the new version later. ### SimplyEdit Our journey away from Ariadne actually started here, in 2015. With Ariadne we've been on the forefront of online WYSIWYG editing ever since Internet Explorer 5.0. We've made an online site builder and editor, named [Vedor](https://vedor.nl) that was well received by end-users, but the business model didn't pan out. Initially we just wanted to extract the WYSIWYG editor out of Ariadne and make it easy to connect to any kind of CMS out there, even WordPress. But we've ended up with a much more general and capable system, that allows for much more than traditional WYSIWYG editing. We're using it as a render system, like Vue, React or Angular. It has two-way databinding. And most importantly, the data model divides content just two basic types: fields and lists. Lists are unique to SimplyEdit. They are the superpower that enables extremely simple website design customization, without programming. All you need is HTML and CSS knowledge to create completely custom building blocks for websites or webapplications. Wordpress (relatively) recently added the [Gutenberg editor](https://wordpress.org/gutenberg/). This allows editors to drag and drop blocks into website content. However, unlike SimplyEdit, you can only drop blocks into specific content fields. The blocks need a lot of PHP code to define and cannot change per field or page. The storage is a hacky extension to HTML. SimplyEdit stores all data as JSON. Fields may contain HTML, which is stored as a string in the JSON data. Each page or component can define its own set of fields and lists, which may be shared or partially shared with other pages or components. We've added a [JSON-CSS search and filter tool](https://github.com/SimplyEdit/json-css-search) to allow you to search the JSON data for specific content. ### SimplyView While building SimplyEdit we found that we really liked the power and (external) simplicity of the two-way databinding for building webapplications. But we missed some application specific tooling. So over the past few years we've gradually added those components. Some of these components could very well be replaced by other components, not made by Muze. But what we focused on, and why we prefer them are: - simplicity: The components do only one thing. They contain minimal code. They connect to HTML through attributes only. - replaceability: Each component is simple enough to be replaced with another implementation. If a simplyview component isn't capable enough, you shouldn't extend it, but replace it. - well integrated: Each component is build as if it was a standard part of the browser. It automatically recognizes and initializes HTML elements, no matter when or how they appear in the DOM. - well behaved: Each component follows the standards, so no custom attributes except for `data-` attributes. Each component uses only namespaced attributes, so `data-simply-*`. The list of SimplyView components is: - [simply.app](simply.app.md): optional app framework that ties the rest together for ease of use. - [simply.route](simply.route.md): simple application routing. - [simply.command](simply.command.md): commands, routes and actions combine to make the API of the frontend. It eases seperation and communication between frontend developers (JavaScript) and frontend designers (HTML, CSS) - [simply.action](simply.action.md) - [simply.collect](simply.collect.md): A declarative way to react to changes in forms. - [simply.activate](simply.activate.md): A declarative way to initialize behaviour for interactive elements. - [simply.include](simply.include.md): A client-side include mechanism. - [simply.api](simply.api.md): Simplify connecting to remote API's. - [simply.viewmodel](simply.viewmodel.md): Allow more complex behaviour in SimplyEdit lists, like paging, sorting, filtering. - [simply.path](simply.path.md): Some common code to work with `.` seperated paths. ### Hosting and API's With Ariadne we've become accustomed to building server-side API's and storage in a way that isn't very common. In fact only [Zope](https://zope.org) (or [Plone](https://plone.org)) share a similar approach. Instead of writing an API front loader with hard-coded routing, as is common, we focus on building a hierarchical data model. This data model has typed objects which may contain other typed objects, according to a well structured hierarchy. Each object type has its own set of functionality in the form of templates. So by building the datastructure, you build the functionality. We call this data-based routing. You don't always need it, but to do it right, each component needs to understand and work with hierarchical, tree based data. All this is contained inside Ariadne, as a monolith. We've been extracting these components since 2013, as ARC components. The current list is: - [`arc/base`](https://github.com/Ariadne-CMS/arc-base/): Common datatypes and functionality shared by all ARC components. Is installed automatically if needed. - [`arc/cache`](https://github.com/Ariadne-CMS/arc-cache/): Cache functionality and a caching proxy class. - [`arc/events`](https://github.com/Ariadne-CMS/arc-events/): Fire events and listen for them in a tree structure, modelled after the browsers DOM events. - [`arc/config`](https://github.com/Ariadne-CMS/arc-config/): Configuration management, storing and retrieving configuration settings in a tree structure. - [`arc/web`](https://github.com/Ariadne-CMS/arc-web/): Simple and correct manipulation of URL's, HTTP Headers and a HTTP client. Also includes a simple intrustion detection component to prevent cross site scripting attacks. - [`arc/xml`](https://github.com/Ariadne-CMS/arc-xml/): Parsing and writing XML made simple. - [`arc/html`](https://github.com/Ariadne-CMS/arc-html/): Parsing and writing HTML also made simple. - [`arc/grants`](https://github.com/Ariadne-CMS/arc-grants/): Access control management in a tree structure, like a filesystem. - [`arc/prototype`](https://github.com/Ariadne-CMS/arc-prototype/): Prototypical inheritance for PHP, like javascript. - [`arc/store`](https://github.com/Ariadne-CMS/arc-store/): Structured (hierarchical) object store, with a simple query DSL. And currently in prototype phase: - [`arc/rest`](https://github.com/poef/arc-rest): a set of cookbook examples on how to use ARC components to build API's. ## Software Philosophy Muze does software a bit different than most other companies. Here are some of the lessons we've learned. We like dynamic languages, not only because of the instant feedback (no compilers) but also because of a few other things: - Respect the boundaries. Your software should fit the problem domain, so the main boundaries are those of the domain, not of the tech stack. Don't merge concepts across those boundaries. But within the problem boundary, also don't cross the tech boundaries. Javascript is for behaviour, HTML is for structure, CSS is for design. - Clean code is not paramount, clean architecture is. Build minimal software, out of minimal components. Components shouldn't be more than about 1000 lines of code, and they should be replacable. How clean those 1000 lines of code are is then largely irrelevant. - Extreme late binding. Your code can change runtime, not only compile time. When handled well, this means less code that does more. It also means unit tests! - Duck typing vs. strict typing. Especially in the 'glue' layer of your application, duck typing means more flexibility. When things get complex, strict typing may be required. Choose wisely. - Datatypes matter. Spend extra time on defining the data types and the operations on them, and the rest will follow. - Architecture follows from the problem domain, not from frameworks or design patterns. Those can help, but are not a starting point. - Abstractions are your job, make sure you fit them well. Leaky abstractions have a way of growing the complexity of software. If requirements force you to create unwieldy, complex or leaky abstractions, try to change the requirements. Any implementation is always a first approximation to an eventual solution, so it doesn't have to handle everything. It should however be a stable base for future growth. - Write code to throw it away, you will eventually anyway. By respecting boundaries (or layers) and keeping components small and general, you should be able to remove a component and replace it with something better/bigger/faster when needed. Don't paint yourself in a corner. Clean up after yourself if you've been hacking stuff. - Don't fight the platform. If the browser has a solution for a problem, use it. Don't implement your own version. Extend the existing one if it is missing features, or skip that feature altogether. Understand the basic concepts and philosophy of the platform. So grok the browser, the webserver and the database. ### Examples Ariadne has a data storage format that is based on the format of the URL. Because it is designed to host web applications, it makes sense to base the core abstractions around the URL. Specifically the path of the URL becomes the path to the data in the object store. Because this is inherently hierarchical (a tree), the object store is also a tree. Then all other abstractions in Ariadne follow this path and tree concept. So events follow the tree, just like the DOM tree in a webbrowser. Grants (access control or rights) follow the tree as well. In ARC we've made the tree concept an explicit data type. SimplyEdit is a web editor and render engine. It tries to do as much as possible inside the browser, instead of the server. The server only needs to store and serve data. This includes resizing images so we can serve them responsively. Using canvas and a smart algorithm SimplyEdit can scale the images down with fidelity. SimplyEdit then selects the best matching size image on rendering the HTML later. This uses the browser srcset standard, but extends it. The default storage engine stores the whole website as a single json file. This is generally good enough and opens the possibility to search through the entire site contents. When the site is too big for this, SimplyEdit allows you to replace the storage engine with your own custom solution.