# Nimble Wrap-Up Blog Post


## Introduction

This is a wrap-up post for the Nimble project work funded through [NGI Zero Core](https://nlnet.nl/core), a fund established by [NLnet](https://nlnet.nl/) with financial support from the European Commission's [Next Generation Internet](https://ngi.eu/) program. This post can be read in isolation, but the [previous post](https://7bindustries.com/blog/nimble_project_1.html) on this project will give some extra context.

As a quick recap, this development team met at a hackathon hosted by the [Open Toolchain Foundation](https://opentoolchain.org/) (OTFN) in March 2023. OTFN exists to "promote the adoption and support the development of tools (software) for designing products, machines, etc. (hardware), namely Open Source software tools for engineering." At that hackathon a prototype was built of a system that created parametric documentation which stayed in sync with its associated CAD, even when changes were made. The prototype focused on the Nimble network rack and worked well, but it had a long way to go to be production-ready. NLnet funded the development of this production-ready system so that Wakoma and the team could bring it to reality. There was also a focus on the part of NLnet to ensure the toolchain was independent of the Nimble project so that it would be useful to a wide range of hardware projects. I feel that the team has done a good job of meeting this goal.

**Side Note**: It has been reported that the Next Generation Internet program is not going to be included in the next EU Horizon funding mandate. NGI and other open source funders are currently trying to figure out what is next for their their funding model if that happens. If you appreciate any of the many projects that NLnet and/or NGI has funded (including this one), please read up on the situation. Links to read more are [here](https://www.theregister.com/2024/07/17/foss_funding_vanishes_from_eus/) and [here](https://www.ow2.org/view/Events/The_European_Union_must_keep_funding_free_software_open_letter).

## The Result

If you want to skip straight to seeing the end result, there is a hosted version of the Nimble Configurator [here](). CadOrchestrator also has a sample cup holder tree project which shows how flexible CadOrchestrator can be in serving other projects. A hosted version of that example project can be found [here](http://cadorchestrator-example.gitbuilding.io/).

A flow chart of this project's architecture was provided in the [previous blog post](https://7bindustries.com/blog/nimble_project_1.html#the_project), and while the end result uses most of the same pieces, there are some key changes in how things are organized. The following updated diagram, created by [Julian Stirling](https://julianstirling.co.uk/), goes into much more detail on the final architecture and how the data flows.


The diagram itself provides some explanation text, and what follows is additional information on the components. It is important to note that the tools mentioned below can stand on their own, apart from the Nimble project. With the exception of the Nimble-specific source files and the hardware database, these tools can be used directly for other projects.

* **[Networking Hardware Database](https://nocodb.wakoma.net/dashboard/#/nc/view/fdd19387-69e5-4b79-966d-65c3733339cd)** - Wakoma did a survey and assembled information on the most common network components used for Nimble racks, and then added that information into this database. The information includes the physical dimensions of the device in each shelf, which allows CAD models and assemblies to be generated in an automated way. This database was created specifically to service the Nimble project, but the information contained in the database could be useful to other projects.
* **[CadOrchestrator](https://gitlab.com/gitbuilding/cadorchestrator)** - The area boxed in as CadOrchestrator was labelled generically as the "orchestration" layer in the original diagram. This application provides a web server and the user interface which allows a user to configure and generate their customized documentation and CAD. It has "orchestrator" in the name because it acts like an orchestra conductor to the rest of the application, triggering actions based on user input.
* **[ExSource Tools](https://gitlab.com/gitbuilding/exsource-tools)** - ExSource is a tool which can run various programs to generate output, kind of like a build tool for hardware projects. For instance, it can be passed a source file for OpenSCAD, told what format to output that CAD file to, and it will run OpenSCAD and convert the file. This is frequently used to convert source files to an output format (i.e. STL). ExSource also has the capability of running cq-cli, which is the next piece of the puzzle.
* **[cq-cli](https://github.com/CadQuery/cq-cli)** - Most of the CAD work done in this project is in [CadQuery](https://cadquery.readthedocs.io/en/latest/), or a wrapper around CadQuery called CadScript. cq-cli is an existing command line tool which takes these files and converts them to an output format (i.e. STL, STEP) for use in the generated documentation. During the course of this project, cq-cli was expanded to handle FreeCAD files as well, including parametric ones. This means that with a single tool, ExSource/cq-cli can convert all CAD files needed in this project to other formats. A special assembly renderer script was also created that allows cq-cli to trigger the output of PNG image renders of Nimble shelf and rack assemblies. This includes annotated and exploded views, which leverage callouts (i.e. assembly lines) to make assembly instructions more expressive and easier to follow.
* **[GitBuilding](https://gitlab.com/gitbuilding/gitbuilding)** - GitBuilding was developed to allow creation of high-quality, easily updated hardware documentation, with a focus on assembly instructions. GitBuilding uses text-based source files which can be manipulated by a program, enabling the sort of automatically adapting documentation which this project requires. GitBuilding ties together all of the source documentation and the automated changes to that source, along with the output CAD files to generate the resulting documentation that the user can see and interact with. As stated above, this documentation is kept up-to-date with the current state of the configuration of the Nimble rack.
* **Source Files** - At the heart of the system is the text-based source files. Some are configuration files that tell CadOrchestrator how to do its job, and others are source files for the resulting CAD and documentation. These source files may be modified by CadOrchestrator based on user input to achieve automated adaptation of the documentation to any given configuration. You can see the flow of where these files are used in the diagram above.

## Reflections on the Result

Creating a truly modular automatic, infinitely configurable hardware project that generates CAD, renders, and documentation on the fly was ambitious.

The team learned valuable lessons during the project. As always with programming, generating something that is simple and flexible is easy enough, as is generating something complex and inflexible. We used this project to create the complexity we needed to build and document the Nimble, and then at regular intervals we paused to refactor it into more self contained and flexible tools and functions.

We have not reached the holy grail of a highly complex and truly flexible system that is usable for any project. Aspects of the codebase were easy to generalise, and these have been generalised into their own tools. Other aspects still need unpacking.

The prototype architecture we have created maps the necessary interconnections. We hope it can be used as a base for a more systematically generalised tool in the future.

## Where to Next?

A great first step would be for other open hardware projects to implement parts (or the whole of) this architecture for their own projects. The separate parts of this toolchain are listed above, but two good high-level project repositories to use as templates and discussion spaces would be the [Nimble](https://github.com/Wakoma/nimble/issues) and [CadOrchestrator-Example](https://gitlab.com/gitbuilding/CadOrchestrator-Example/-/issues) repositories. These also provide entry points for individuals who would like to contribute to this framework.

There are many options for where the development of this project could go next. Below are a few ideas.

* The hardware documentation tools that we have demonstrated require the user to be Python-savy enough to install packages on their local machines. Many hardware engineers find this difficult. Creating a hosted documentation platform that allows anyone to suggest edits to hardware documentation would be a huge step forward for open hardware. We would like to do this in a way where we do not hold a user's data, we just help you manipulate it. One option would be to support the GitHub and GitLab APIs, a hosted service would then create Merge Requests with updated documentation.
* Better tools for visualising assembly steps. We already have a demo (slider on 3rd view here https://gitbuilding.gitlab.io/gb3d/) of how to create animated assemblies. However, the workflow to create the correct animated files is prohibitive to using this in any evolving open hardware project.
* Expand the generated renders of CAD assemblies to include numeric callouts that correlate to a Bill of Materials, and that update automatically with changes in the assemblies or BOM.

## Thanks

Two organizations were directly involved in making this project happen.

* [NLnet](https://nlnet.nl/) - Provided the funding that allowed for this project to be built. The funding was specifically provided by [NGI Zero Core](https://nlnet.nl/core), a fund established by NLnet with financial support from the European Commission's [Next Generation Internet](https://ngi.eu/) program.
* [Wakoma](https://wakoma.co/) - Provided the point-of-contact with NLnet, and guidance on what the community needed in the final result of this project. One of the forms this took was to establish and populate the hardware database mentioned above. Wakoma also provided developer operations expertise to host the hardware database that feeds into the documentation/CAD system, as well as providing hosting for the finished server framework.

## Team

Thanks goes out to several individuals as well. The following people were members of the team for this project.

* **[Eric Nitschke](https://github.com/Wakoma/)** - Project management, database development and community interface.
* **[Julian Stirling](https://julianstirling.co.uk/)** - Project management, GitBuilding expertise and [Jack of all trades](https://en.wikipedia.org/wiki/Jack_of_all_trades).
* **[Antonio de Jesus Anaya Hernandez](https://kny5.github.io/personal_portfolio)** - DevOps Engineer
* **[Amudhan Manivasagam](https://github.com/Wakoma/)** - Developer operations (DevOps) and database development.
* **[Andreas Kahler](https://github.com/drayde)** - CAD model development
* **[Jeremy Wright](https://7bindustries.com/)** - CAD assembly development, CAD rendering and annotation.

## Acknowledgements

The following people provided editorial feedback on this blog post, and it is much appreciated.

* [Eric Nitschke](https://github.com/Wakoma/)
* [Julian Stirling](https://julianstirling.co.uk/)
* [Andreas Kahler](https://github.com/drayde)