Update 11: Addressing Implementation Challenges

Introduction

In this eleventh update, I'll walk you through the challenges I encountered with the log-based implementation and the approach I've taken to overcome them.

The Challenge

I recently faced a hurdle while working on the WAL implementation. The issue was that the SQLite3 library I was using did not expose the functions required for independent use of the WAL implementation. This prompted me to explore alternative solutions.

Exploring Possible Solutions

In my pursuit of a solution, I brainstormed various possibilities. I'll reiterate the options I considered:

1. Forking nim-sqlite3-abi

   The idea here was to fork the nim-sqlite3-abi repository and manually add the necessary functions to the header file. However, this approach posed a concern. The vendor/nim-sqlite3-abi would have a new git pointer, which raised compatibility issues.

2. Considering Redis AOF (Append-Only File)

   Another option I considered was using Redis AOF (Append-Only File). While this approach seemed promising, there were challenges. The AOF.c file lacked a header file, and it had numerous dependencies that my project didn't require. Adapting it to meet my specific needs would involve substantial edits.

Guidance from Zahary

I shared my concerns and ideas with Zahary, and he responded by saying that extracting the relevant code into a small, stand-alone library seemed like a reasonable approach.

The Current Approach

Currently, I've taken the following steps:

I forked the nim-sqlite3-abi repository, rebranding it as nim-sqlite3-wal. I'm in the process of separating, exporting, and wrapping the necessary write ahead log functions, ensuring they meet the specific requirements of my project and separating them as a stand-alone library.