Writing A React-Admin Data Provider For Offline-First Applications

I created a localForage dataProvider for react-admin. You can find it at marmelab/ra-data-localforage. Creating a data provider is actually quite straightforward, as I'll explain in this post.

Why LocalForage Is A Better API For Offline-First

To build desktop applications, toolkits like Electron.js let us use frontend JS frameworks. But that comes at a price: startup time, runtime performance, and app size are subpar compared to native desktop apps. A new Rust toolkit for desktop apps named Tauri was recently released, and I wanted to test it.

One thing led to another and I added react-admin to this project. My goal was to make a completely offline application. For that, I needed a local storage mechanism. React-admin has a provider that can use window.localStorage. However, as Peter LePage points out in this article, localStorage blocks the main thread, it is limited in size, and it can only contain strings.

On the other hand, the localForage library allows storing data asynchronously in the browser, leveraging IndexedDB. LocalForage even provides a fallback to WebSQL or localStorage when IndexedDB isn't available. That seemed like the perfect storage for my app.

To use LocalForage in a react-admin app, I needed a dataProvider. So I thought: Let's build it!

From Disk To Memory

As Alexis pointed out in a previous article about an SQLite Data Provider for react-admin, react-admin's data providers map a predefined API for querying data (getOne, getList, etc.) to a backend API.

LocalForage provides the storage mechanism, but not the sophisticated querying abilities of a REST API. In order not to reinvent the wheel, I decided to use FakeRest, which as its name indicates simulates a fake REST server in memory based on JSON data. So localForage provides the persistent storage, but the queries to the dataProvider are handled live by FakeRest.

I opted for registering each resource individually, by prefixing it with ra-data-local-forage-[ResourceName]. This avoids storing one big blob for the entire database, which is preferable if localForage falls back to WebSQL or localStorage.

The first step of my provider was to load the data from localForage.

const prefixLocalForageKey = 'ra-data-local-forage-';

const getLocalForageData = async (): Promise<any> => {
    const keys = await localforage.keys();
    const keyFiltered = keys.filter(key => {
        return key.includes(prefixLocalForageKey);
    });

    if (keyFiltered.length === 0) {
        return undefined;
    }

    const localForageData: Record<string, any> = {};
    for (const key of keyFiltered) {
        const keyWithoutPrefix = key.replace(prefixLocalForageKey, '');
        const res = await localforage.getItem(key);
        localForageData[keyWithoutPrefix] = res || [];
    }
    return localForageData;
};

// I use await to be sure that when I launch my application I have the data displayed
const localForageData = await getLocalForageData();
const data = localForageData ? localForageData : defaultData; // users can add default data if they want (JSON format)

With that data, I could then use FakeRest to build an in-memory database:

const baseDataProvider = fakeRestProvider(
    data,
    loggingEnabled
) as DataProvider;

Mapping Data Provider Methods

React-admin expects every dataProvider to return an object with 9 methods:

• getList: to get a list of resources
• getOne: to get a single resource
• getMany: to get a list of resources by ids
• getManyReference: to get a list of resources by reference ids
• update: to update a resource
• updateMany: to update a list of resources
• create: to create a resource
• delete: to delete a resource
• deleteMany: to delete a list of resources

FakeRest has a read API that is close enough to this list to make the dataProvider read methods just proxies.

getOne: <RecordType extends RaRecord = any>(
    resource: string,
    params: GetOneParams<any>
) => baseDataProvider.getOne<RecordType>(resource, params),

The write methods required a bit more work. Here is the create case for example:

create: <RecordType extends RaRecord = any>(
    resource: string,
    params: CreateParams<any>
) => 
    // we need to call fakerest first to get the generated id
    baseDataProvider
      .create<RecordType>(resource, params)
      .then((response) => {
          // update the in-memory data object
          if (!data.hasOwnProperty(resource)) {
            data[resource] = [];
          }
          data[resource].push(response.data);
          // update the local storage
          localforage.setItem(`${prefixLocalForageKey}${resource}`, data[resource]);
          return response;
      }),

Note that localForage.setItem is async, so it will not block the thread. As the "live" data provider is FakerRest, the dataProvider doesn't need to wait for localStorage to reply. And no need to stringify the data save, unlike with localStorage.

The process of writing the custom dataProvider went smoothly and took about one hour.

Using An Asynchronous Data Provider

Unlike other available providers, the loading of data in the IndexedDB makes my data provider asynchronous. I want to display my application only once the data is loaded.

import * as React from "react";
import { Admin, Resource } from 'react-admin';
import localForageDataProvider from 'ra-data-local-forage';

import { PostList } from './posts';

const App = () => {
  const [dataProvider, setDataProvider] = React.useState<DataProvider | null>(null);

  React.useEffect(() => {
    async function startDataProvider() {
      const localForageProvider = await localForageDataProvider();
      setDataProvider(localForageProvider);
    }

    if (dataProvider === null) {
      startDataProvider();
    }
  }, [dataProvider]);

  // hide the admin until the data provider is ready
  if (!dataProvider) return <p>Loading...</p>;

  return (
    <Admin dataProvider={dataProvider}>
      <Resource name="posts" list={ListGuesser}/>
    </Admin>
  );
};

export default App;

Conclusion

Writing a dataProvider for react-admin was less complicated than I expected. Using FakeRest to simulate a REST API helped a lot to simplify this work.

My dataProvider offers a great alternative to ra-data-localstorage, as it supports larger databases. It's now referenced in the official list of data providers in the react-admin documentation.

This data provider helped me build a robust offline-first desktop application. Thanks to GreenFrame, we know that this kind of architecture emits less CO2 than always online apps.

If this article made you want to create your own dataProvider, please publish it on npm with an open-source license, as I did, and reference it on the react-admin documentation.