Hello! It’s been about six months since I had the initial idea for Beryl. I’ve been having a blast working on bringing it to life, but almost none of it is worth anybody actually trying to use. So you get words instead! This is part update, part design document.

If you’re not familiar with Beryl, it’s Obsidian meets Todoist. Check out the initial concept post.

Here’s the progress I’ve made in the last half-year:

Beryl-go

The weekend after my initial concept, I threw together a go library that could parse a file with Beryl tasks. It’s very brittle, but a good-sized test suite let me throw something together that tested my ideas. All it does is take a text file and returns an object hierarchy of tasks. It fully crashes if it encounters a parsing error, so it’s not my favorite thing ever!

Beryl-TUI

Shortly after, I made a bare-bones TUI using BubbleTea.

BubbleTea is awesome, but gosh proper TUI development is hard! I think it would be entirely impossible without BubbleTea, but the process didn’t really mesh with my brain. I learned a lot, and I’d love to revisit this with a more robust parser. But it was time to move on to a GUI.

GUI

For Beryl to be useful for me, I want to be able to use it on every device I have. So that means supporting:

• Windows
• Mac
• Linux
• Web
• Android
• iOS

as well as being able to sync tasks between them.

As much as I would love to learn the true native language for all of those platforms, I have no desire to! I love hybrid technologies, so my next step was to find tech that would both accomplish my goals and be fun to work on.

Capacitor

I’ve been using capacitor for mobile app development for a while now, and I really like it! Instead of shipping all of chromium with every app, capacitor uses the device webview. I also like that capacitor only deals the the borders between the web app and the native app. Both of them are free to be modified by you. I’d go on to apply this philosophy of decoupling to all of the rest of Beryl.

Wails

I dreaded the idea of using Electron for the desktop app. I really wanted something that would let me use the device webview again. Amazingly, Wails exists! Wails lets me write desktop applications that use the system webview, all in golang! This is a decoupled tool, so a perfect choice!

Svelte

I’ve used Vue and Angular in the past, and both of them were not exactly pleasant to work with. Angular is not very decoupled, and Vue does a lot of magic stuff that I would rather avoid. The final nail in the coffin though was trying both Angular and Vue in Wails and Svelte. Both took 5ish seconds to load up the web app after the native app launched, which just sucks. So something faster was needed.

While I was learning how to use CouchDB, I stumbled across this git repo which said:

Pssst… I’m not working with Vue anymore. This plugin is old and bad. Here is a better alternative: https://github.com/MDSLKTR/pouch-vue. However, if for whatever reason you find yourself looking at this, please re-evaluate your life choices and look at Svelte.

So I did! Svelte is very, very nice to work with so far. It has an advantage over other SPA frameworks, because it does a lot of pre-compilation at build time, instead of when the user loads the app. That got the capacitor/wails start time down to around one second!

Sync

I’ve only worked on typical CRUD apps until this point. I really need sync to be robust, which means I shouldn’t be the one writing my own sync service from scratch!

This took a surprisingly long time to figure out how best to approach this. I considered:

• Abusing git somehow
  • Pros: history baked in
  • Cons: no encryption, dealing with merges isn’t straightforward, not what git is for
• Embedded rclone
  • Pros: peer-to-peer
  • Cons: also not really intended to be shipped inside, say, an ios app
• FileCloud??
  • Cons: proprietary, api seems bad
  • Pros: none

After thrashing around for a few weeks, I finally found Obsidian LiveSync! This is great, because it’s open source, and already behaves a lot like what I’d like to do. Digging into the source code, it uses CouchDB.

Putting all these pieces together, I was able to make a test website, desktop app, and mobile app that syncs a text field!

Putting it all together

At this point, I started on a very basic desktop app using all of these bits. Amazingly, it works!

All it does is one project at a time, and frankly, it looks bad. But I feel like the concept has been proven!

New logo

At this point, I’m feeling more and more confident that this project is going to actually happen. So I went ahead and commissioned a talented graphic designer to make a Beryl Logo!

I LOVE this logo. I don’t know how else to gush about it. I also made a project mastodon account at beryl@fosstodon.orgif you want to be kept up to date!

After this logo was finished, I saw an ad for used beer barrels, so of course I had to pick one up.

Monorepo

By this point, Beryl has 5 independent projects. I’d never worked on something with so many moving parts, and it’s kinda a pain in the butt! Keeping track of all the different commands and running them in the right order really took the fun out of playing with beryl. So I decided it was monorepo time!

My hope was to find a build system that would work on unix and windows, and would manage builds across languages. Ideally I wanted to be able to run build desktop or build android and not have to remember what commands and what dependencies need to be pulled in.

NX

NX seemed to fit the bill right away. Wide platform and language support, and a large, active community! The appeal of automatically recognizing dependencies was very powerful.

But trying it out was not a pleasant experience. None, and I mean none, of the language generators worked. Like I would generate entire test repos with their tools, and they wouldn’t work correctly. Also, core features silently failed! I wasn’t looking for it at the outset, but cached builds sound great! And NX would just not cache, and eventually when I did get it working I had no idea what I did to fix it. Not a good first impression.

Earthly

While trying to get NX working, I came across a lot of documentation for NX on Earthly’s website. It was obviously content marketing, but good content marketing! It had lots of useful information. I’m a sucker for good documentation, so I tried Earthly next. Immediately, the idea of needing docker or podman did not thrill me, but isolated builds make sense. Unfortunately, I was not able to get podman working on my computer, so I pretty much immediately rage quit Earthly,

Please

Please caught my eye during this whole process. Being written with golang in mind is a huge plus, but I didn’t think it was a good idea because it lacked windows support. After my experience so far, I’ll settle for good unix support. Hopefully I can do e2e testing on windows via WSL.

So I gave please a whirl. It’s great! It has a really steep learning curve since the builds are hermetic. They don’t happen in a container, but they are run in a temporary directory, in a minimal shell environment with tightly controlled environment variables. Dependencies also have to be explicitly configured, which is kinda a bummer, but I think it will make a system that is a lot more robust.

The whole monorepo setup is not entirely done, but I think I have a handle on please.

The Future

And that brings us to today!

Going forward I have two big initiatives for Beryl: Gnesis and Foundation.

In real life, gnesis is a mineral that beryl is sometimes found inside. Gnesis is my name for the current Beryl app using the not-so-great go parser. Whenever I get tired of working on the longer-term Foundation I’ll implement actual features in Gnesis. I know that if I go awhile without doing something that I can really use, I’ll lose motivation.

Foundation is the 1.0 version of Beryl. I realized one big problem with todo lists is everyone has a slightly different, unique workflow that they really like. So I envision Beryl being very extensible, with a plugin system and themes.

I have no idea how the plugins and themes will work. I know there is a standard ‘plugin architecture’ pattern, but I certainly don’t know it. I have a couple of tutorials to follow though, so we’ll see where I end up after I do those!

Also beryl-go needs re-worked. I need to understand how parser-combinators work first so I can actually add new features. If I wanted to add proper error handling or syntax highlighting or support for things like titles/sections, I don’t think I could do that in beryl-go, I’m too scared to touch it! So the next step there will be to rewrite it as a JavaScript library, so it can be used on all devices.

I’m hoping that a parser that produces an Abstract Syntax Tree will let me implement the VSCode Language Server Standard. That would be a huge win, because I could include a text editor within Beryl itself that has syntax highlighting, and provide fix suggestions for errors.

I’m also surprised at the lack of off-the-shelf file watchers! I need a way to recursively watch a folder for changes. This won’t be too bad on the golang side, but I’ll have to make a new capacitor plugin for android and iOS.

Finally, I want to really lean into automated testing. My big weakness is spotting bugs before they happen. I know that when fixing bugs, I’m liable to re-introduce old bugs without noticing. So I’m hoping to use Appium, and to get in the habit of making end-to-end tests for new features and bug fixes.

Conclusion

It’s been a busy six months! I do wish that this was the kind of project that was easier to release in tiny, incremental pieces. But from what I’ve learned, it’s very difficult to add a plugin architecture into software after the initial design process, so I have to get that ready now.

I do feel some trepidation about the complexity of this project. I initially described Beryl as ‘helper software’, and this is very much not that. It’s grown into a very ambitious project that might genuinely take another six months before it’s really ready. I have the drive to simplify as much as possible, and I do wonder if this is really as simple as I could go with Beryl. I guess the great thing is that the complex, slick app doesn’t have to be the only thing in this ecosystem. You could hack something together in HyperCard that’s Beryl complaint, and use both!

The important thing is I’m having a great time on this project. I’m learning so much about interesting technology, and I really believe that I’m making something useful! At the minimum, it’s the todo list I’ve always wanted.

If you’re interested in learning more, please feel free to follow the project at @beryl@fosstodon.org or follow the Beryl rss feed. Thanks for reading!