Shortwave Blog

Shortwave is reinventing what’s possible with email, starting with an inbox that keeps you organized by default.

Real-time React apps using Watchables

January 6

5 min read

Have you ever needed to build a user interface in React that updates in real-time based on server events? We struggled to find a good pattern that made this easy in our app. After trying to do this using Redux, we eventually found a much better way. In this post I’ll document our journey and our (open source!) solution.

Shortwave’s real-time UI

A major goal of Shortwave is to provide a much more real-time email experience. Users should see new emails right away without needing to click to refresh, and triage actions taken on one device should update other devices (and tabs) immediately.

To accomplish this, our apps have a websocket connection that incrementally syncs down data from our backend. We store that data locally and merge it with local state based on user actions, so that we can compensate for latency and give users a responsive app even when their network isn’t.

Our apps have client-side logic in TypeScript for managing this local state, including handling asynchronous server updates, user input, disk persistence, and other business logic. To make our user interface work, however, we need to get this state into our React components.

Why Observables didn’t work

Our first attempt at doing this was to just expose the state as an Observable to get it into React. Let’s take an example of our draft service - which has an interface like this:

interface DraftService { watchDraft(draftId: DraftId): Observable<Draft>; watchAllDrafts(): Observable<Record<DraftId, Draft>>; }

Usage of our service looks something like this:

const DraftPreview: React.VFC<{draftId: DraftId}> = ({draftId}) => { const service: DraftService = useDraftService(); const [draft, setDraft] = useState<Draft | null>(null); useEffect(() => { const observable = service.watchDraft(draftId); const subscription = observable.subscribe(setDraft); return () => subscription.unsubscribe(); }, [service, draftId]); return draft === null ? 'Loading...' : `Draft: ${draft.subject}`; };

This worked, but we quickly noticed a problem. The first render pass in React always resulted in displaying the loading state to the user. Even if we had a draft loaded in memory and we could display it instantly, we had to wait until the useEffect hook ran to update the state. For toy apps, this isn't noticeable because useEffect can run and React can rerender the component before the browser has the chance to paint. Our application was large enough that there was flickering of the loading state every time a component was mounted.


Why Redux didn't work

Our fix for flickering? Put the state into Redux! So now our app had a hook like this at the top of the component hierarchy:

function useSyncDraftsIntoRedux() { const service = useDraftService(); const dispatch = useDispatch(); useEffect(() => { const observable = service.watchAllDrafts(); const subscription = observable.subscribe( (drafts) => dispatch(setDraftsAction(drafts)) ); return () => subscription.unsubscribe(); }, [service, dispatch]); }

This lets us fix our component’s flickering while simplifying it at the same time. Great!

const DraftPreview: React.VFC<{draftId: DraftId}> = ({draftId}) => { const draft = useSelector( (state: Store) => state.drafts.drafts[draftId] ); return draft == null ? 'Loading...' : `Draft: ${draft.subject}`; };

What’s not to love? Well, a lot it turns out! First of all, it's not clear when to load data in this model. In our draft example we can start piping them all into Redux at app load time, but we can't do this with all of our data. We have to manually set up the sync into Redux anytime we display the data anywhere in the app - a cumbersome and bug-prone pattern. We also ran into performance issues early on and needed to optimize our selectors with tools like Reselect.

Additionally, we have all our state duplicated into two places - first the service and then the Redux store. Not only did this make it difficult to figure out where the source of truth for our state was, it also required a bunch of extra code. We needed to create reducers to handle the state and actions so we can wire up the service to the store. We also were doing this before Redux Toolkit was production ready, which just meant even more boilerplate code to write.

Overall our use of Redux felt like overkill and imposed a very rigid structure to our code - all we needed was a way to expose our application state to React!

Watchables to the rescue

We wanted a simpler solution. We liked the simplicity of the observables pattern, but observables are designed for streams of data and don’t necessarily have a “current” value. We needed to synchronously access state for our first render, so what we wanted was a data structure that both holds a current value and has a notification mechanism for when it’s updated. Enter Watchables - a small data structure we built for exactly this purpose - to expose a value into React. At its core, Watchables have a small API that looks something like:

/* A readonly value that can be watched. */ interface Watchable<T> { /* If a watchable has a value or is empty (a loading state). */ hasValue(): boolean; /* Access the current value. */ getValue(): T; /* * Watch for updates to the value. * Will initially be fired with the current value if there is one. */ watch((value: T) => void): Unsubscribe; }; type Unsubscribe = () => void; /* A mutable watchable value. */ interface WatchableSubject<T> extends Watchable<T> { update(value: T): void; }

We can now update our component to look something like the following:

const DraftPreview: React.VFC<{draftId: DraftId}> = ({draftId}) => { const service: DraftService = useDraftService(); const watchable = useMemo( () => service.watchDraft(draftId), [service, draftId] ); const [draft, setDraft] = useState<Draft | null>( watchable.getOrDefault(null) ); useEffect(() =>, [watchable]); return draft === null ? 'Loading...' : `Draft: ${draft.subject}`; };

Watchables also have some other nice properties - they allow for an empty loading state, frequently updated values can be snapshotted, and you can do memoized state transformations. Combining these with a small set of hooks allowed us to simplify our component even more:

const DraftPreview: React.VFC<{draftId: DraftId}> = ({draftId}) => { const service: DraftService = useDraftService(); const draft = useMemoizedWatchable( () => service.watchDraft(draftId), [service, draftId] ); return draft === null ? 'Loading...' : `Draft: ${draft.subject}`; };

We now use Watchables all over in our application, for loading and displaying messages, drafts, contact information, and online presence status. It's become a fundamental part of our application and has helped us to simplify our architecture and define clear boundaries between our business logic and user interface.

As part of this blog post, we’ve open sourced our implementation of Watchables along with a small set of hooks at Adding it to your application is as simple as

npm install --save @shortwave/watchable
. More information can be found on GitHub. If you found this post interesting, check us out - we're hiring!

Read more
Email: the future of messaging

September 21, 2021

4 min read


Last year, a team of ex-Firebasers and I started Shortwave, a stealth startup we’re unveiling today. With Shortwave, we’re placing all of our chips on one big bet: email will dominate the future of messaging.

We believe that email, despite all of its flaws, will eventually win as the way we communicate online, displacing iMessage, WhatsApp, Slack, WeChat, and every other messaging app in your pocket today.

At this point, you may be thinking “Email?! Isn’t that the past?”, and you wouldn’t be alone. The email ecosystem has stagnated as the world changed around it. Email inboxes are overwhelmed with a volume of automated messages they were never designed to handle. Email clients — hamstrung by decades-old protocols and UI concepts — have failed to take advantage of mobile. Meanwhile, messaging apps have raced ahead with real-time updates, native support for groups, easy sharing of photos and videos, end-to-end encryption, and more.

However, email still has two key advantages that outweigh all of its flaws:

  1. It’s universal — With nearly 4 billion users, email is the most ubiquitous messaging technology and the most reliable way to communicate with just about anyone.
  2. It’s open and decentralized — Developers can build interoperable services without asking anyone for permission. This allows users to choose from a wide variety of clients and hosting providers, as well as a flourishing ecosystem of tools built on top of the protocol (for sending newsletters, wedding invitations, invoices, and more). Most importantly, if you’re unhappy with your provider, you can switch to another one or even run your own servers.

Messaging apps, on the other hand, are closed services that don't interoperate. You can’t send messages between Slack and Teams or from WhatsApp to iMessage, so you end up juggling a dozen apps to stay connected. These apps are controlled by a select few companies that mandate the use of their own clients, force all network traffic through their servers, and restrict what services can be built on top of them. These centralized services also create a single point of failure and control, making them unreliable platforms for free expression when it matters most. If you’re unhappy with your experience, too bad.

The future of online communications cannot be trusted to a centralized service. It must be built on a foundational technology that is:

  • Universal — I should be able to talk to anyone in the world from a single app.
  • Open & decentralized — I should be able to develop and run my own clients, servers, and services.
  • Intelligent — I should never miss an important message, no matter how many messages I receive in a day.
  • Flexible & expressive — I should be able to communicate the way I want, whether that means writing a long memo or sending a quick emoji reaction to a friend’s video.
  • Private & secure — I should feel confident sending even my most sensitive information.

The right choice isn’t to invent a new protocol. The right choice is to build on email — a technology that is already universal, open, decentralized, and battle-tested for over 40 years.

While email today does not yet satisfy all of our requirements, recent advancements in machine learning, encryption, and decentralized governance have made some of email’s most challenging problems far more tractable. People and businesses are also reconsidering social conventions around email and messaging due to the shift to remote work, making this an ideal time for a new approach. With the right investments in modern clients, new servers, upgraded protocols, and thoughtful design, it’s now possible to build a user experience that lives up to email’s potential.

Upgrading email will be a difficult, multi-year journey, but we have the right team, resources, and determination to play the long game. Our first product is a brand new inbox experience for your existing Gmail account. It is still being tested privately, but we are providing early access to a select set of individuals and companies. If you would like to try it out, request access here. If our mission excites you and you want to help build the next wave of communication, we’re looking for great people to join us, so please get in touch.

Read more

Sign up for our newsletter

Hear about the latest updates and opportunities with Shortwave!