Commit Graph

400 Commits

Author SHA1 Message Date
Ericson "Fogo" Soares
81c294356a [ENG-794] Library deletion hangs (#1137)
Solving deadlock on library deletion
Also improving some minor stuff on thumbnails remover actor
2023-07-27 23:18:45 +00:00
jake
874faa4e55 [ENG-926] Prevent thumbnail destruction and fix the remover (#1127)
* fix(core): thumbnail removal

* chore(core): add todo

* New actor on steroids

* Improving thumbnail remover actor

* Ignoring errors from files that doesn't exist

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-07-25 14:21:50 +00:00
Ericson "Fogo" Soares
9e65a22e74 [ENG-875] Re-index & Re-thumb should be smart (#1130)
* New sub_path rescan rspc route

* Changing some rescan call sites to use new route
2023-07-25 12:47:56 +00:00
Brendan Allan
40669b768d Push libraries into vec after loading (#1129)
push libraries into vec after loading
2023-07-24 17:17:02 +00:00
Brendan Allan
9907ac29dc CRDTOperation receiving (#1122)
* operation receive + compare

* cleanup + deduplication

* operation receive + compare

* cleanup + deduplication

* sync route + operation grouping

* tag assign sync

* proper relation support in sync debug page

* migration

* separate core-sync + utils crates

* separate p2p event loop from manager

* cleanup library handling

* clippy

* feature gate sync messages properly

* make migration not add required field
2023-07-24 15:26:00 +00:00
Oscar Beaumont
86e4e8bffc [ENG-925] - Arc<Library>/Arc<NodeContext> (#1124)
* `Arc<Library>` + `Arc<NodeContext>`

* fix specta version

* remove unneccessary derefs

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 10:59:57 +00:00
Ericson "Fogo" Soares
783fbd2a53 [ENG-778] Location clean up job (#1123)
* Introducing thumbnail remover actor
Also tweaking the orphan remover actor

* Rust fmt
2023-07-20 15:17:54 +00:00
Oscar Beaumont
c025326e03 [ENG-906] Initial library sync (#1095)
* ffs

* typo

* yeet library data over p2p

* fix a bunch of edge cases

* report complete status on responder

* better log

* fix types

* mobile debug screen

* mobile + P2P is a mess

* feature flag mobile p2p pairing

* wrong one

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-17 11:53:25 +00:00
Brendan Allan
63a1648904 Use codegen for ModelSyncType exec (#1098)
* use codegen for ModelSyncType exec

* folder client format

* hande create operation properly
2023-07-15 04:22:04 +00:00
Brendan Allan
cf30d14148 SQLite connection limit (#1111)
connection_limit=1
2023-07-15 03:56:00 +00:00
Oscar Beaumont
97a7104473 [ENG-537] Notification system (#805)
* `Notification` struct

* Manage dem notifications

* a dash of rspc

* `read` field

* `handle_notification_callback`

* some Prisma

* TS-centric notifications backend

* `useNotifications` hook

* Notification test buttons
2023-07-12 07:43:27 +00:00
Oscar Beaumont
74bf175442 [ENG-759] P2P Cleanup (#1062)
* less stupid name

* yeet

* awaiting futures is kinda important lol

* no-async

* more proto stuff

* cleanup

* move it round

* None of my homies like broadcast

* minor

* do the shuffle

* restore after force push

* reusing `sysinfo::System` as intended

* fix lol

* remove `node_id` from `Volume`

* Remove `node_local_id` from `Library`

* Remove `Job` to `Node` relation

* feature flags be like

* press save 4head

* remove `Location` -> `Node` relation

* `volume.rs` to `volume/mod.rs`

* yes

* add `Instance` model and deprecate `Node` model

* pairing is better now

* Pretty code

* thinking in code

* wip

* What if Observables but in Rust?!

* Observables aren't it + `useP2PEvents` hook

* more around some jsx

* Trade offer: bad code for working pairing?

* Unit test pairing protocol

* fix everything up

* corrections

* Have you got a moment for our lord and saviour Clippy

* tsc --fixMyCode

* Prisma being wacky

* Config files being f'ed up

* broken config after migrating

* Zed being Zed

* Argh

* cliipzy

* rewind

* Fix truncate logic

* wip: instances in peer metadata

* Rethink instance ids

* fix

* whoops

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:23:30 +00:00
Niklas Wojtkowiak
53dd42780f Add functionality to some Overview Catrgories (#1090)
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:01:30 +00:00
Brendan Allan
4240048d33 [ENG-846] Preferences (#1047)
* preferences table + trait

* cleaner implementation

* router + ts type

* preference test

* per-column list view preferences

* preference read + write

* add some docs

* migration + docs

* remove ts expect error
2023-07-12 04:32:53 +00:00
Ericson "Fogo" Soares
8f97756c29 [ENG-884] Update file_path metadata (#1077)
* Partitioned paths to be updated on walker

* Updating file_paths in indexer

* Properly tracking modified date on renames

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-11 15:14:11 +00:00
nikec
cdf1767e85 [ENG-885] Move online locations to lib store (#1074)
* Enable trace logging for invalidate query

* Fixing quick rescan on explorer
And annotating wrong quick rescan usage on top bar reload button

* Move online locations to lib store

* Fix topbar reload path

* fix mobile

* Remove unused imports + format
 - Adjust `arraysEqual` arguments type

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-05 22:27:38 +00:00
Vítor Vasconcellos
dac54e44d1 [ENG-765] Reported Total capacity and Free space are wrong (#1066)
* Attempt at fixing stats

* Fix macOS disk stats retrieve logic
 - Ignore mounted dmgs when calculation disk total/free size
 - Only take into account disk mounted by macOS

* macos only import

* Fix Linux

* Replace byte-size with a custom implementation that supports BigInt

* Fix NaN in Statistics

* clippy

* fmt

* Move linux get_volumes to a specilized function
 - Fix ZFS handling
 - Improve handling of disk symlinks and multiple mounts

* Fix macOS
2023-07-05 18:22:56 +00:00
Matthias Berchtold
31e659bf18 Indexer rule No Git (#1048)
feat: no git indexer

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-07-03 11:45:36 +00:00
Brendan Allan
e8f88bfc49 Search filter cleanup (#1063)
cheeky cleanup
2023-07-03 03:02:57 +00:00
Ericson "Fogo" Soares
9a6d88a096 [ENG-850] Wrongly labelled .ts files (as video) (#1052)
* Fixing conflits of mts code files showing as video

* Introducing a reidentify_objects param on fullRescan

* Fix job output metadata in case of completed with errors
2023-06-29 02:58:33 +00:00
Ericson "Fogo" Soares
b5bd5471e7 [ENG-823] Fixing bad handling of dots in directory name (#1051)
* Fixing bad handling of dots in directory name

* fix ci not running on rust

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-28 21:41:55 +00:00
Ericson "Fogo" Soares
3a46ea4370 Job System and Watcher improvements (#1030)
* Trying to make watcher more resilient between creates and updates

* More checks on create vs update

* Rust fmt

* PCR rev

* Windows CI fix

* Some indexer improvements

* Builder pattern for jobs and reports

* fix list view invalid date

* fix progress bar

* Normalizing job metadata output

* fixed job metadata

* Removing inner report struct from identifier job

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-06-28 16:30:19 +00:00
Brendan Allan
a84eec36c5 [ENG-806] Better library name handling (#1043)
better library name handling
2023-06-28 09:11:24 +00:00
Utku
8e69728589 Fix job resuming/pausing and cancelling (#1035)
why this works?
2023-06-27 19:26:59 +00:00
Oscar Beaumont
cc6a83f011 Job System - Simplify data (#1017)
* yeet

* destructure `JobState` when passed to `finalize`.
Let him cook.

* `impl Job for JobInitData` instead of it being `Job::Init`
The pan is on high heat and the 3 sh-meats are coming out.

* Remove `JobInitData` & `IntoJob` & `Job::new`

* clean up a little

* Remove `Job.stateful_job` in favor of `Job.state.init`
Yeah this isn't great but lets get these changes working and then think again about it.

* Properly call `StatefulJob.finalize`

* she'll be right

* fix `initialize_resumable_job`

* builder syntax for creating jobs

* Have Rust gone too far?

* Minor fixes

* fixes

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-06-27 07:15:49 +00:00
Oscar Beaumont
de7e4df0b4 [ENG-823] Fix some stuff (#1024)
* SQLite journal bug

* Remove "job.report" indexes thing

* resume button - job not found?

* panic hook with logging

* it did

* large pain

* fix `check_nested_location`

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-27 05:12:04 +00:00
Oscar Beaumont
d759a7f9e1 Job System - Better Progress (#1015)
yeet

Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-06-26 17:49:55 +00:00
Gedeon
8af9c49d5f [ENG-803] Set date upon tag creation (#1021)
* set date_created to tag

* update tag date modified when edited

* use `FIxedOffset` for tag `date_created`

* remove dead code and use fixedoffset

* discard duplicate code

* remove dead code & add current time to field

* remove unused code

* no longer needed in data

* set creation time for tags

* fix tag creation time

* fix & finalize timestamps

* `update_many` was not needed here

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-06-25 13:44:21 +00:00
Ericson "Fogo" Soares
05bbfa7d92 [ENG-670 / ENG-774] Resumable and Pausable jobs / Queued jobs don't show in the job manager (#961)
* Fixing some warnings

* Optimizing job workers

* Removing the delay from getting job reports

* remove some commented out stuff

* Trying to optimize job report route

* fix thread 'tokio-runtime-worker' panicked at 'attempt to subtract with overflow'

* fix progress bar

* Now pause works from the UI

* Fix

* Now the worker set job report to paused

* show errors for job pause/resume

* bunch 'o ui fixes

* order location in the sidebar

* fix some text

* fix clear all jobs

* fix clear jobs & job group ui

* show queued jobs

* object validator job text

* make cancel button work

* better executable logo

* Now cancel button works instantly

* disable resume button

* remove disabled props from pause/resume buttons

* remove large comments

* show paused progress

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-24 07:23:30 +00:00
Oscar Beaumont
94467c495e [ENG-792] App no starty (#1004)
* lul, message parsing with a response is cringe

* deadlocks hurt me physically

* super secret commit with many secret secrets

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-23 06:22:24 +00:00
Ericson "Fogo" Soares
9fd0d9ab90 [ENG-770 / ENG-773] Copy/paste / Explorer context menu duplicate (#992)
* Fixing copy, cut and paste

* Bug on duplicating files without extensions

* Fix paste only ignoring the location origin when validating relative paths
 - Add Error handling to all context menu actions
 - Hide FS actions in Overview and MediaView
 - Remove redundant useExplorerSearchParams
 - Add default schema to the Zod param hooks

* Hide FS actions in all Explorers but location ones
 - Fix react warnings in RenameTextBox
 - Fix ReactTextBox not selecting the whole file name, instead of just up to file extension
 - Fix react warning due to inputing missing onChange

* basic show jobs for cut, copy and delete in job manager

* tweaks

* Fixing some warnings and minor tweaks

* Rust fmt

* Fix cut between locations on Linux

* Change how routes retrieve route params
 - Replace useZodRouteParams with useLoader
 - Define all schemas in the router file instead of in the route Component
 - Parse schema in loader to avoid requiring the route Components having to import the schema from the router file

* Remove default argument from useZodRouteParams

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: James Pine <ijamespine@me.com>
2023-06-22 23:41:29 +00:00
Brendan Allan
7f9648c467 [ENG-782] Seeded tags (#994)
* seeded tags infra

* dumb

* here's my seed

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-22 12:08:38 +00:00
jake
ad1f99cf19 [ENG-737] Location deletion hanging (#978)
* fix concurrency issue with lib deletion

* clear only completed jobs

* slight fixes

* experimental but faster i think

* lower chunk size
2023-06-22 10:49:21 +00:00
Oscar Beaumont
49321cf50f [ENG-783] rename node (#993)
rename node + hide some stuff
2023-06-22 08:35:09 +00:00
Jamie Pine
f1bf7fba35 [ENG-779] Finalize UI (#986)
* [ENG-779] Finalize UI
This is one branch with a variety of UI changes

add tag select mode bar without functionality 

fix group job status 

add notice icon with info to stat icons

add WIP notice to media view 

add modal before add location with greyed out clouds

remove disappearing add location button

add WIP spacedrop page 

bring back limited key manager UI 

add options bar on search page without functionality 

Add greyed out encrypt library button or setting

See more button on locations

Show locations on node screen

Fix overview category left padding

* key manager placeholder

* stat info

* nodes screen

* location click yay

* fix size in bytes

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>

* small ui improvements

* sh*tty see more button

* last touches

* fix merge boo boo

* Fix mobile
 - Move `getItemObject`, `getItemFilePath`, `getItemLocation`, `getExplorerItemData` to @sd/core to allow mobile to use them

* Formatting

* Normalize displayed file size between all screens
 - Replace every use of internal formatBytes with byte-size dep

---------

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-06-22 06:34:45 +00:00
Ericson "Fogo" Soares
40fa435368 [ENG-768 / ENG-769] Regen thumbnails / Generate checksums (#983)
* Making sub_path api for jobs harder to misuse

* re-enable context menu options

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-21 12:55:19 +00:00
Brendan Allan
244a8e0440 Better tag unassign (#989)
better tag unassign
2023-06-21 12:22:20 +00:00
jake
36650f63e0 Remove useless clone from mutation (#988)
* remove useless clone from mutation

* fix clippy too
2023-06-21 10:53:49 +00:00
jake
b93f60bedc Fix tag removal (#985)
fix tag removal
2023-06-21 08:47:59 +00:00
Oscar Beaumont
4398640057 [ENG-685] Sqlite Multithreaded (#917)
* pragma moment

* Try socket_timeout?

* bye

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 20:09:52 +00:00
Ericson "Fogo" Soares
cca390d593 [ENG-743 / ENG-744] Duplicate item / Delete explorer item (#977)
* Fixing a bunch of small bugs related to FS

* cleanup

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 09:44:13 +00:00
Brendan Allan
f708679ad8 Do all category logic on server (#972)
* do all category logic on server

* cleanup
2023-06-19 12:21:32 +00:00
Brendan Allan
cf7f3c6f59 Fix categories queries (#971)
* more stringent hidden filter check

* add missing migration case

* don't do sync stuff when sync disabled

* remove old stuff
2023-06-19 11:28:07 +00:00
Oscar Beaumont
d70c24db49 ordering for create library is wrong (#970) 2023-06-19 09:01:56 +00:00
Oscar Beaumont
dc84dd384d [ENG-732] Pairing by library (#943)
* library keypair

* allow opening stream to non-connected node + more

* library identity

* fix types

* fix maybe undefined

* don't forgor migrations file

* library manager inside p2p manager

* rename

* `NodeInformation` struct

* node info exchange

* fill in info

* streamify tunnel

* use tunnel for p2p events

* libp2p is annoying + stop leaking private key's

* Clippy cleanup
2023-06-19 05:13:30 +00:00
Brendan Allan
771aaccc89 Replace some default fields (#968)
replace some default fields
2023-06-19 00:48:38 +00:00
Brendan Allan
7a9ec4aedf Remove owned sync (#967)
* remove owned sync + cleanup

* no more atomic records

* byebye owned

* remove owned ops from schema
2023-06-18 23:45:06 +00:00
Brendan Allan
a6be04a5e5 Make indexer rules optional (#965)
* optional indexer rules

* lockfile
2023-06-18 22:13:55 +00:00
Oscar Beaumont
7ddca26e57 [ENG-568] Spacedrop Progress (#956)
* `Transfer` struct

* progress moment

* fixes
2023-06-18 08:40:51 +00:00
Jamie Pine
68d4b7ceda [ENG-747] Indexer rule for Apple Photos .photoslibrary (#963)
photo library indexer rule + memorized video playback volume
2023-06-17 10:33:04 +00:00