mirror of
https://github.com/spacedriveapp/spacedrive.git
synced 2026-04-19 14:08:45 -04:00
144 lines
4.8 KiB
Plaintext
144 lines
4.8 KiB
Plaintext
---
|
||
title: Normalised Cache
|
||
index: 12
|
||
---
|
||
|
||
# Normalised Cache
|
||
|
||
We use a normalised cache for our frontend to ensure the UI will always contain consistent data.
|
||
|
||
## What this system does?
|
||
|
||
By normalising the data it's impossible to get "state tearing".
|
||
|
||
Each time `useNodes` or `cache.withNodes` is called all `useCache` hooks will reexecute if they depend on a node that has changed.
|
||
|
||
This means the queries will always render the newest version of the model.
|
||
|
||
## Terminology
|
||
|
||
- `CacheNode`: A node in the cache - this contains the data and can be identified by the model's name and unique ID within the data (eg. database primary key).
|
||
- `Reference<T>`: A reference to a node in the cache - This contains the model's name and unique ID.
|
||
|
||
## High level overview
|
||
|
||
We turn the data on the backend into a list of `CacheNode`'s and a list of `Reference<T>`'s and then return it to the frontend.
|
||
|
||
We insert the `CacheNode`'s into a global cache on the frontend and then use the `Reference<T>`'s to reconstruct the data by looking up the `CacheNode`'s.
|
||
|
||
When the cache changes (from another query, invalidation, etc), we can reconstruct *all* queries using their `Reference<T>`'s to reflect the updated data.
|
||
|
||
|
||
## Rust usage
|
||
|
||
The Rust helpers are defined [here](https://github.com/spacedriveapp/spacedrive/blob/main/crates/cache/src/lib.rs) and can be used like the following:
|
||
|
||
```rust
|
||
pub struct Demo {
|
||
id: String,
|
||
}
|
||
|
||
impl sd_cache::Model for Demo {
|
||
// The name + the ID *must* refer to a unique node.
|
||
// If your using an enum, the variant should show up in the ID (although this isn't possible right now)
|
||
fn name() -> &'static str {
|
||
"Demo"
|
||
}
|
||
}
|
||
|
||
let data: Vec<Demo> = vec![];
|
||
|
||
// We normalised the data but splitting it into a group of reference and a group of `CacheNode`'s.
|
||
let (nodes, items) = libraries.normalise(|i| i.id);
|
||
|
||
// `NormalisedResults` or `NormalisedResult` are optional wrapper types to hold a one or multiple items and their cache nodes.
|
||
// You don't have to use them, but they save declaring a bunch of identical structs.
|
||
//
|
||
// Alternatively add `nodes: Vec<CacheNode>` and `items: Vec<Reference<T>>` to your existing return type.
|
||
//
|
||
return sd_cache::NormalisedResults { nodes, items };
|
||
```
|
||
|
||
## Typescript usage
|
||
|
||
The Typescript helpers are defined [here](https://github.com/spacedriveapp/spacedrive/blob/main/packages/client/src/cache.tsx).
|
||
|
||
### Usage with React
|
||
|
||
We have helpers designed for easy usage within React's lifecycle.
|
||
|
||
```ts
|
||
const query = useLibraryQuery([...]);
|
||
|
||
// This will inject all the models into the cache
|
||
useNodes(query.data?.nodes);
|
||
|
||
// This will reconstruct the data from the cache
|
||
const data = useCache(query.data?.item);
|
||
|
||
console.log(data);
|
||
```
|
||
|
||
### Vanilla JS
|
||
|
||
These API's are really useful for special cases. In general aim to use the React API's unless you have a good reason for these.
|
||
|
||
```ts
|
||
const cache = useNormalisedCache(); // Get the cache within the react context
|
||
|
||
// Pass `cache` outside React (Eg. `useEffect`, `onSuccess`, etc)
|
||
|
||
const data = ...;
|
||
|
||
// This will inject all the models into the cache
|
||
cache.withNodes(data.nodes)
|
||
|
||
// This will reconstruct the data from the cache
|
||
//
|
||
// *WARNING* This is not reactive. So any changes to the nodes will not be reflected.
|
||
// Using this is fine if you need to quickly check the data but don't hold onto it.
|
||
const data = useCache(query.data?.item);
|
||
|
||
console.log(data);
|
||
```
|
||
|
||
## Design decisions
|
||
|
||
### Why `useNodes` and `useCache`?
|
||
|
||
This was done to make the system super flexible with what data you can return from your backend.
|
||
|
||
For example the backend doesn't just have to return `NormalisedResults` or `NormalisedResult`, it could return:
|
||
|
||
```rust
|
||
pub struct AllTheData {
|
||
file_paths: Vec<Reference<FilePath>>,
|
||
locations: Vec<Reference<Location>>,
|
||
nodes: Vec<CacheNode>
|
||
}
|
||
```
|
||
|
||
and then on the frontend you could do the following:
|
||
|
||
```ts
|
||
const query = useQuery([...]);
|
||
useNodes(query.data?.nodes);
|
||
const locations = useCache(query.data?.locations);
|
||
const filePaths = useCache(query.data?.file_paths);
|
||
```
|
||
|
||
This is only possible because `useNodes` and `useCache` take in a specific key, instead of the whole `data` object, so you can tell it where to look.
|
||
|
||
|
||
## Known issues
|
||
|
||
### Specta support
|
||
|
||
Expressing `Reference<T>` in Specta is really hard so we [surgically update](https://github.com/spacedriveapp/spacedrive/blob/a315dd632da8175b47f9e0713d3c7fc470329352/core/src/api/mod.rs#L219) it's [type definition](https://github.com/spacedriveapp/spacedrive/blob/a315dd632da8175b47f9e0713d3c7fc470329352/crates/cache/src/lib.rs#L215).
|
||
|
||
This is done using `rspc::Router::sd_patch_types_dangerously` which is a method specific to our fork [spacedrive/rspc](https://github.com/spacedriveapp/rspc).
|
||
|
||
### Invalidation system integration
|
||
|
||
The initial implementation of this idea with an MVP. It works with the existing invalidation system like regular queries, but the invalidation system isn't aware of the normalised cache like a better implementation would be.
|