svelte • rust • electron • napi

Implementing A Svelte Store In Rust

By Dave Ceddia

We’re going to build a Svelte store, written in Rust, and run it in an Electron app.

This spun out of a video editor which I originally built in Swift. It turns out people on Windows want to edit videos too, so now I’m rewriting it with Rust, Electron, and Svelte with an eye for performance.

Rust is a systems programming language, akin to C or C++, but safer. It’s known for being super fast.

Electron is a framework for building cross-platform desktop apps with HTML, CSS, and JavaScript. It’s known for being kinda slow and bloaty. But there’s an ace up its sleeve: Electron apps can be extended with compiled native code, and if you do the heavy stuff in native code you can a nice speed boost.

Svelte is a JavaScript UI framework – an alternative to React, Vue, Angular, or one of the other 7500 frameworks. Svelte uses a compiler to generate small, fast, reactive code.

I figured by combining them, and doing most of the heavy lifting in Rust, I’d end up with an app that feels snappy.

The full, completed project is on GitHub and it has instructions on how to run it, along with my rollercoaster of a commit history while I was trying to get it working.

Here’s what it looks like:

How Svelte Stores Work

One of the things I love about Svelte is its reactivity model, and in particular, its concept of stores. A store is a reactive variable that holds a single value.

Any part of the app can subscribe to the store, and every subscriber will be (synchronously!) notified when the store’s value is changed.

Here’s a simple example (live version here):

<script>
  import { onDestroy } from 'svelte';
  import { writable } from 'svelte/store';
  
  // Make a store
  const count = writable(0);

  // Subscribe to it, and update the displayed value
  let visibleCount = 0;
  const unsubscribe = count.subscribe(value => {
    visibleCount = value;
  });

  function increment() {
    // Replace the store's value with (value + 1)
    count.update(n => n + 1);
  }

  // Tidy up when this component is unmounted
  onDestroy(unsubscribe);
</script>

<button on:click={increment}>Increment</button>
<p>Current value: {visibleCount}</p>

You click the button, it updates. Nothing too mindblowing. But this is just the “low-level” API.

It looks a lot nicer when you introduce Svelte’s special reactive store syntax with the $ (try the live example):

<script>
  import { onDestroy } from 'svelte';
  import { writable } from 'svelte/store';
  
  // Make a store
  const count = writable(0);

  function increment() {
    $count += 1;
  }
</script>

<button on:click={increment}>Increment</button>
<p>Current value: {$count}</p>

It does the exact same thing, just with less code.

The special $count syntax inside the <p> is setting up a subscription behind the scenes, and updating that specific DOM element when the value changes. And it handles the unsubscribe cleanup automatically.

There’s also the $count += 1 (which can also be written $count = $count + 1). It reads like plain old JavaScript, but after the value is changed, this store will notify all of its subscribers – in this case that’s just the $count in the HTML below.

The Svelte docs have a great interactive tutorial on stores if you want to learn more.

The Important Thing is the Contract

It’s easy to look at code like this and assume it’s all magic, especially when there’s fancy syntax like $store.

There’s a chunk of data-intensive code that I wrote in JS instead of Rust because I had this mindset of, “I want the reactivity so it has to be in JavaScript”.

But if you take a step back and look at the underpinnings of how the magic actually works, sometimes you can find new and interesting ways to extend it!

Svelte stores were designed well to allow this: they follow a contract.

The short version is that in order to be a “Svelte store,” an object needs:

  • A subscribe method that returns an unsubscribe function
  • A set method if you want to make it writable
  • It must call the subscribers synchronously (a) at subscribe time and (b) any time the value changes.

If any JS object follows these rules, it’s a Svelte store. And if it’s a Svelte store, it can be used with the fancy $store syntax and everything!

Calling Rust From JavaScript

The next piece of this puzzle is to write some Rust code that can be exposed as an object in JavaScript.

For this, we’re using napi-rs, an awesome framework for connecting Rust and JavaScript together. The creator, LongYinan aka Broooooklyn is doing amazing work on it, and the latest updates (in v2) have made the Rust code very nice to write. Here’s a taste, the “hello world” of Rust functions:

#[macro_use]
extern crate napi;

/// import the preludes
use napi::bindgen_prelude::*;

/// annotating a function with #[napi] makes it available to JS,
/// kinda like `export { sum };`
#[napi]
pub fn sum(a: u32, b: u32) -> u32 {
  a + b
}

Then in JavaScript, we can do this:

// Hand-wavy pseudocode for now...
// The native module has its own folder and
// build setup, which we'll look at below.
import { sum } from './bindings';

console.log(sum(2, 2)) // gives correct answer

A Boilerplate Project with Electron, Rust, and Svelte

We’ve got the big pieces in mind: Electron, Svelte stores, Rust that can be called from JS.

Now we just need to… actually wire up a project with 3 different build systems. Hoorayyyyy. I hope you can hear the excitement in my voice.

So, for this prototype, I took the lazy way out.

It’s a barebones Electron app with the Svelte template cloned into one subfolder, and the native Rust module in another (generated by the NAPI-RS CLI).

The dev experience (DX) is old-school: quit the whole app, rebuild, and restart. Sure, some sort of auto-building, auto-reloading Rube Goldberg-esque tangle of scripts and configuration would’ve been neat, but I didn’t wanna.

So it has this mile-long start script that just cd’s into each subfolder and builds it. It’s not pretty, but it gets the job done!

  "scripts": {
    "start": "cd bindings && npm run build && cd .. && cd ui && npm run build && cd .. && electron .",
    "start:debug": "cd bindings && npm run build:debug && cd .. && cd ui && npm run build && cd .. && electron .",
    "start:clean": "npm run clean && npm run start:debug",
    "clean": "cd bindings && rm -rf target"
  },

We’re not going for awesome DX here. This is a prototype. Awesome DX is Future Work™.

Beginning To End: How It Works

Personally I really like to trace the execution from the very first entry point. I think it helps me understand how all the pieces fit together. So here’s the chain of events that leads to this thing working, with the relevant bits of code:

1. You run npm start. It builds everything, and then runs electron .

package.json
"start": "cd bindings && npm run build && cd .. && cd ui && npm run build && cd .. && electron .",

2. Electron finds and executes main.js because package.json tells it to (via the main key)

package.json
{
  "name": "electron-quick-start",
  "version": "1.0.0",
  "description": "A minimal Electron application",
  "main": "main.js",
  ...
}

3. main.js spawns a BrowserWindow, and loads up index.html

main.js
function createWindow() {
  // Create the browser window.
  const mainWindow = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      contextIsolation: true,
      // Preload will make the native module available
      preload: path.join(__dirname, 'preload.js')
    }
  })

  // Load the index.html of the app.
  mainWindow.loadFile('index.html')
}

4. main.js also specifies a preload.js, where you’re allowed to expose native modules. This is where the Rust module is imported and exposed as window.Napi. (See Security below)

preload.js
// Make native bindings available to the renderer process
window.Napi = require('./bindings');

5. index.html loads the Svelte app’s JavaScript that was built in Step 1

index.html
<html>
  ...
  <body>
    <!-- You can also require other files to run in this process -->
    <script src="./ui/public/build/bundle.js"></script>
  </body>
</html>

6. Svelte has its own ui/main.js, which imports and creates the App component, and mounts it at document.body.

ui/main.js
import App from './App.svelte';

const app = new App({
  target: document.body,
});

export default app;

7. App.svelte instantiates our Rust store with an initial value, which calls the constructor in Rust.

ui/App.svelte
<script>
  import Counter from "./Counter.svelte";
  let showCounter = true;
  let counter = new Napi.Counter(42);
</script>

8. Since Svelte needs to render the counter, it immediately calls .subscribe with a callback, which calls subscribe in Rust.

ui/public/build/bundle.js [compiled by Svelte]
function instance($$self, $$props, $$invalidate) {
  let $counter;
  let showCounter = true;
  let counter = new Napi.Counter(42);
  component_subscribe($$self, counter, value => $$invalidate(1, $counter = value));
  const click_handler = () => $$invalidate(0, showCounter = !showCounter);
  const click_handler_1 = () => set_store_value(counter, $counter = Math.floor(Math.random() * 1234), $counter);
  return [showCounter, $counter, counter, click_handler, click_handler_1];
}

9. The subscribe function, according to the contract, needs to immediately call the provided callback with the current value, so it does that, and then saves the callback for later use. It also returns an unsubscribe function that Svelte will call when the component is unmounted.

#[napi]
impl Counter {
  // ...

  #[napi]
  pub fn subscribe(
    &mut self, env: Env, callback: JsFunction
  ) -> Result<JsFunction> {
    // Create a threadsafe wrapper.
    // (to ensure the callback doesn't 
    // immediately get garbage collected)
    let tsfn: ThreadsafeFunction<u32, ErrorStrategy::Fatal> = callback
      .create_threadsafe_function(0, |ctx| {
        ctx.env.create_uint32(ctx.value).map(|v| vec![v])
      })?;

    // Call once with the initial value
    tsfn.call(self.value, ThreadsafeFunctionCallMode::Blocking);

    // Save the callback so that we can call it later
    let key = self.next_subscriber;
    self.next_subscriber += 1;
    self.subscribers.borrow_mut().insert(key, tsfn);

    // Pass back an unsubscribe callback that
    // will remove the subscription when called
    let subscribers = self.subscribers.clone();
    let unsubscribe = move |ctx: CallContext| -> Result<JsUndefined> {
      subscribers.borrow_mut().remove(&key);
      ctx.env.get_undefined()
    };

    env.create_function_from_closure("unsubscribe", unsubscribe)
  }
}

Security: Electron and contextIsolation

Electron is split into 2 processes: the “main” one (which runs Node, and is running main.js in our case) and the “renderer”, which is where your UI code runs. Between the two sits the preload.js. Electron’s official docs explain the process model in more detail.

There are a few layers of security in place to prevent random scripts from gaining unfettered access to your entire computer (because that would be bad).

The first is the nodeIntegration flag, defaulted to false. This makes it so that you can’t use Node’s require() inside the renderer process. It’s a little annoying, but the upside is that if your Electron app happens to open (or is coerced into opening) a sketchy script from somewhere, that script won’t be able to import Node modules and wreak havoc.

The second is the contextIsolation flag, which defaults to true. This makes the preload script, which runs inside the renderer, can’t access window and therefore can’t expose any sensitive APIs directly. You have to use the contextBridge to expose an API that the renderer can use.

Why am I telling you all this? Well, if you look at the preload.js example above, you’ll see that it sets window.Napi directly. It’s not using contextBridge, and contextIsolation is disabled in this project. I tried turning it on, but evidently constructors can’t be passed through the bridge. There might be an alternative way to solve this – if you know of one please let me know!

If your app doesn’t load external resources, and only loads files from disk, my understanding is that leaving contextIsolation disabled is ok.

I’m writing this as a proof of concept WITH THE CAVEAT that this is less secure than it could be (if you have ideas for improvement, let me know on Twitter).

How the Rust Works

The short answer is: it follows the Svelte store contract :) Let’s see how.

It all happens in one file, bindings/src/lib.rs.

First, there’s a struct to hold the current value of the counter, along with its subscribers.

I don’t think the ThreadsafeFunctions can be compared for equality, so I put them in a map instead of a vector, and used the next_subscriber to hold an incrementing key to store the subscribers.

#[napi]
pub struct Counter {
  value: u32,
  subscribers: Rc<RefCell<HashMap<u64, ThreadsafeFunction<u32, ErrorStrategy::Fatal>>>>,
  next_subscriber: u64,
}

Then there are a few functions implemented on this struct. There’s the constructor, which initializes a Counter with no subscribers:

#[napi]
impl Counter {
  #[napi(constructor)]
  pub fn new(value: Option<u32>) -> Counter {
    Counter {
      value: value.unwrap_or(0),
      subscribers: Rc::new(RefCell::new(HashMap::new())),
      next_subscriber: 0,
    }
  }

And there are increment and set functions that do nearly the same thing. Of the two, set is special in that it’s the one that makes this store “writable” in the eyes of Svelte. When we write $count = 7 in JS, that will ultimately call into set here.

#[napi]
pub fn increment(&mut self) -> Result<()> {
  self.value += 1;
  self.notify_subscribers()
}

#[napi]
pub fn set(&mut self, value: u32) -> Result<()> {
  self.value = value;
  self.notify_subscribers()
}

After modifying the value, those functions call notify_subscribers. This one doesn’t have the #[napi] annotation, which means it won’t be callable from JS. This iterates over the subscribers and calls each one with the current value.

Because self.subscribers is an Rc<RefCell<...>> we need to explicitly borrow() it before iterating. This borrow happens at runtime, as opposed to the usual compile-time borrow-checking done by Rust. If something else has this borrowed when we try to borrow it here, the program will panic (aka crash).

I’m reasoning this is panic-free because both the notify_subscribers and the subscribe (the other place that borrows this variable) are running in the single JS main thread, so it shouldn’t be possible for them to step on each others’ access.

fn notify_subscribers(&mut self) -> Result<()> {
  for (_, cbref) in self.subscribers.borrow().iter() {
    cbref.call(self.value, ThreadsafeFunctionCallMode::Blocking);
  }
  Ok(())
}

Most of the real work happens inside subscribe. There are some comments, but also some subtleties that took me some time to figure out.

First, it wraps the callback with a ThreadsafeFunction. I think the reason this works is that ThreadsafeFunction internally sets up a reference counter around the callback. I tried without this at first, and it turned out that the callback was getting garbage-collected immediately after subscribing. Despite storing the callback (and making Rust happy about its ownership), attempting to actually call it was failing.

The ErrorStrategy::Fatal might look alarming, but the alternative, ErrorStrategy::CalleeHandled, doesn’t work at all here. The CalleeHandled style uses Node’s callback calling convention, where it passes the error as the first argument (or null). That doesn’t match Svelte’s store contract, which only expects a single argument. The Fatal strategy passes the argument straight through.

The create_threadsafe_function call itself has a lot going on. The closure being passed in |ctx| { ... } will get called whenever we run .call() on the threadsafe function. The closure’s job is to take the value you pass in and transform it into an array of JavaScript values. So this closure takes the u32 value, wraps it in a JsNumber with create_uint32, and then puts that in a vector. That vector, in turn, gets spread into the arguments to the JS callback.

Saving the callback is important so we can call it later, so self.subscribers.borrow_mut().insert(key, tsfn); does that. We need the borrow_mut because we’re doing runtime borrow checking here.

I initially went with borrow checking at compile time, but the unsubscribe closure threw a wrench in the works. See, we need to add something to the hashmap at subscribe time, and we need to remove something from the same hashmap at unsubscribe time. In JS this is a piece of cake. In Rust, because of the way ownership works, only one thing can “own” self.subscribers at a time. If we moved it out of self and into the unsubscribe closure, then we couldn’t add any more subscribers, or notify them.

The solution I found was to wrap the HashMap with Rc<RefCell<...>>. The Rc part means that the innards can be shared between multiple owners by calling .clone(). The RefCell part means we can mutate the internals without having to pass the borrow checker’s strict rules about mutation. The tradeoff is that it’s up to us to make sure we never have overlapping calls to .borrow() and .borrow_mut(), or the program will panic.

#[napi]
impl Counter {
  // ...

  #[napi]
  pub fn subscribe(
    &mut self, env: Env, callback: JsFunction
  ) -> Result<JsFunction> {
    // Create a threadsafe wrapper.
    // (to ensure the callback doesn't 
    // immediately get garbage collected)
    let tsfn: ThreadsafeFunction<u32, ErrorStrategy::Fatal> = callback
      .create_threadsafe_function(0, |ctx| {
        ctx.env.create_uint32(ctx.value).map(|v| vec![v])
      })?;

    // Call once with the initial value
    tsfn.call(self.value, ThreadsafeFunctionCallMode::Blocking);

    // Save the callback so that we can call it later
    let key = self.next_subscriber;
    self.next_subscriber += 1;
    self.subscribers.borrow_mut().insert(key, tsfn);

    // Pass back an unsubscribe callback that
    // will remove the subscription when called
    let subscribers = self.subscribers.clone();
    let unsubscribe = move |ctx: CallContext| -> Result<JsUndefined> {
      subscribers.borrow_mut().remove(&key);
      ctx.env.get_undefined()
    };

    env.create_function_from_closure("unsubscribe", unsubscribe)
  }
}

That about wraps this up!

I hope I conveyed that this took a good chunk of time fiddling around and running into dead ends, and I’m not sure if I did this the “right” way or if I just happened to stumble into something that works. So please let me know if you have any ideas for improvements. Pull requests welcome :)