Commit Graph
401 Commits
Author SHA1 Message Date
Ericson "Fogo" Soares 26e3b53b21 Small fix due to how sharding works (#1138) 2023-07-28 03:46:44 +00:00
Ericson "Fogo" Soares aec0c94d62 [ENG-794] Library deletion hangs (#1137)
Solving deadlock on library deletion
Also improving some minor stuff on thumbnails remover actor
2023-07-27 23:18:45 +00:00
jakeandEricson Soares 7215e40a3b [ENG-926] Prevent thumbnail destruction and fix the remover (#1127)
* fix(core): thumbnail removal

* chore(core): add todo

* New actor on steroids

* Improving thumbnail remover actor

* Ignoring errors from files that doesn't exist

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-07-25 14:21:50 +00:00
Ericson "Fogo" Soares 9283e74d85 [ENG-875] Re-index & Re-thumb should be smart (#1130)
* New sub_path rescan rspc route

* Changing some rescan call sites to use new route
2023-07-25 12:47:56 +00:00
Brendan Allan 447069388b Push libraries into vec after loading (#1129)
push libraries into vec after loading
2023-07-24 17:17:02 +00:00
Brendan Allan 043b607ad4 CRDTOperation receiving (#1122)
* operation receive + compare

* cleanup + deduplication

* operation receive + compare

* cleanup + deduplication

* sync route + operation grouping

* tag assign sync

* proper relation support in sync debug page

* migration

* separate core-sync + utils crates

* separate p2p event loop from manager

* cleanup library handling

* clippy

* feature gate sync messages properly

* make migration not add required field
2023-07-24 15:26:00 +00:00
Oscar BeaumontandBrendan Allan 23b0c403be [ENG-925] - Arc<Library>/Arc<NodeContext> (#1124)
* `Arc<Library>` + `Arc<NodeContext>`

* fix specta version

* remove unneccessary derefs

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 10:59:57 +00:00
Ericson "Fogo" Soares 03e71e98a4 [ENG-778] Location clean up job (#1123)
* Introducing thumbnail remover actor
Also tweaking the orphan remover actor

* Rust fmt
2023-07-20 15:17:54 +00:00
Oscar BeaumontandBrendan Allan 53ab3178c2 [ENG-906] Initial library sync (#1095)
* ffs

* typo

* yeet library data over p2p

* fix a bunch of edge cases

* report complete status on responder

* better log

* fix types

* mobile debug screen

* mobile + P2P is a mess

* feature flag mobile p2p pairing

* wrong one

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-17 11:53:25 +00:00
Brendan Allan 0a1d721e58 Use codegen for ModelSyncType exec (#1098)
* use codegen for ModelSyncType exec

* folder client format

* hande create operation properly
2023-07-15 04:22:04 +00:00
Brendan Allan f9b32f5a4a SQLite connection limit (#1111)
connection_limit=1
2023-07-15 03:56:00 +00:00
Oscar Beaumont 1a5d2898c2 [ENG-537] Notification system (#805)
* `Notification` struct

* Manage dem notifications

* a dash of rspc

* `read` field

* `handle_notification_callback`

* some Prisma

* TS-centric notifications backend

* `useNotifications` hook

* Notification test buttons
2023-07-12 07:43:27 +00:00
Oscar BeaumontandBrendan Allan cf39f8dbcc [ENG-759] P2P Cleanup (#1062)
* less stupid name

* yeet

* awaiting futures is kinda important lol

* no-async

* more proto stuff

* cleanup

* move it round

* None of my homies like broadcast

* minor

* do the shuffle

* restore after force push

* reusing `sysinfo::System` as intended

* fix lol

* remove `node_id` from `Volume`

* Remove `node_local_id` from `Library`

* Remove `Job` to `Node` relation

* feature flags be like

* press save 4head

* remove `Location` -> `Node` relation

* `volume.rs` to `volume/mod.rs`

* yes

* add `Instance` model and deprecate `Node` model

* pairing is better now

* Pretty code

* thinking in code

* wip

* What if Observables but in Rust?!

* Observables aren't it + `useP2PEvents` hook

* more around some jsx

* Trade offer: bad code for working pairing?

* Unit test pairing protocol

* fix everything up

* corrections

* Have you got a moment for our lord and saviour Clippy

* tsc --fixMyCode

* Prisma being wacky

* Config files being f'ed up

* broken config after migrating

* Zed being Zed

* Argh

* cliipzy

* rewind

* Fix truncate logic

* wip: instances in peer metadata

* Rethink instance ids

* fix

* whoops

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:23:30 +00:00
Niklas WojtkowiakandBrendan Allan c877c03b63 Add functionality to some Overview Catrgories (#1090)
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:01:30 +00:00
Brendan Allan 07c00a8b7f [ENG-846] Preferences (#1047)
* preferences table + trait

* cleaner implementation

* router + ts type

* preference test

* per-column list view preferences

* preference read + write

* add some docs

* migration + docs

* remove ts expect error
2023-07-12 04:32:53 +00:00
Ericson "Fogo" SoaresandVítor Vasconcellos 7cd00dab49 [ENG-884] Update file_path metadata (#1077)
* Partitioned paths to be updated on walker

* Updating file_paths in indexer

* Properly tracking modified date on renames

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-11 15:14:11 +00:00
9d800eec31 [ENG-885] Move online locations to lib store (#1074)
* Enable trace logging for invalidate query

* Fixing quick rescan on explorer
And annotating wrong quick rescan usage on top bar reload button

* Move online locations to lib store

* Fix topbar reload path

* fix mobile

* Remove unused imports + format
 - Adjust `arraysEqual` arguments type

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-05 22:27:38 +00:00
Vítor Vasconcellos f83f29c720 [ENG-765] Reported Total capacity and Free space are wrong (#1066)
* Attempt at fixing stats

* Fix macOS disk stats retrieve logic
 - Ignore mounted dmgs when calculation disk total/free size
 - Only take into account disk mounted by macOS

* macos only import

* Fix Linux

* Replace byte-size with a custom implementation that supports BigInt

* Fix NaN in Statistics

* clippy

* fmt

* Move linux get_volumes to a specilized function
 - Fix ZFS handling
 - Improve handling of disk symlinks and multiple mounts

* Fix macOS
2023-07-05 18:22:56 +00:00
Matthias BerchtoldandUtku 52c5d1d2e9 Indexer rule No Git (#1048)
feat: no git indexer

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-07-03 11:45:36 +00:00
Brendan Allan f1239d166a Search filter cleanup (#1063)
cheeky cleanup
2023-07-03 03:02:57 +00:00
Ericson "Fogo" Soares dcd071d229 [ENG-850] Wrongly labelled .ts files (as video) (#1052)
* Fixing conflits of mts code files showing as video

* Introducing a reidentify_objects param on fullRescan

* Fix job output metadata in case of completed with errors
2023-06-29 02:58:33 +00:00
Ericson "Fogo" SoaresandUtku Bakir 5d49c8a74a [ENG-823] Fixing bad handling of dots in directory name (#1051)
* Fixing bad handling of dots in directory name

* fix ci not running on rust

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-28 21:41:55 +00:00
1dce6380fe Job System and Watcher improvements (#1030)
* Trying to make watcher more resilient between creates and updates

* More checks on create vs update

* Rust fmt

* PCR rev

* Windows CI fix

* Some indexer improvements

* Builder pattern for jobs and reports

* fix list view invalid date

* fix progress bar

* Normalizing job metadata output

* fixed job metadata

* Removing inner report struct from identifier job

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-06-28 16:30:19 +00:00
Brendan Allan 444a6d23c0 [ENG-806] Better library name handling (#1043)
better library name handling
2023-06-28 09:11:24 +00:00
Utku bc100f2505 Fix job resuming/pausing and cancelling (#1035)
why this works?
2023-06-27 19:26:59 +00:00
Oscar BeaumontandEricson Soares d5ddfcc667 Job System - Simplify data (#1017)
* yeet

* destructure `JobState` when passed to `finalize`.
Let him cook.

* `impl Job for JobInitData` instead of it being `Job::Init`
The pan is on high heat and the 3 sh-meats are coming out.

* Remove `JobInitData` & `IntoJob` & `Job::new`

* clean up a little

* Remove `Job.stateful_job` in favor of `Job.state.init`
Yeah this isn't great but lets get these changes working and then think again about it.

* Properly call `StatefulJob.finalize`

* she'll be right

* fix `initialize_resumable_job`

* builder syntax for creating jobs

* Have Rust gone too far?

* Minor fixes

* fixes

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-06-27 07:15:49 +00:00
Oscar BeaumontandUtku Bakir f86b6c8b52 [ENG-823] Fix some stuff (#1024)
* SQLite journal bug

* Remove "job.report" indexes thing

* resume button - job not found?

* panic hook with logging

* it did

* large pain

* fix `check_nested_location`

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-27 05:12:04 +00:00
Oscar BeaumontandEricson Fogo Soares ac5f6323a5 Job System - Better Progress (#1015)
yeet

Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-06-26 17:49:55 +00:00
Gedeonandbrxken128 5a790ac996 [ENG-803] Set date upon tag creation (#1021)
* set date_created to tag

* update tag date modified when edited

* use `FIxedOffset` for tag `date_created`

* remove dead code and use fixedoffset

* discard duplicate code

* remove dead code & add current time to field

* remove unused code

* no longer needed in data

* set creation time for tags

* fix tag creation time

* fix & finalize timestamps

* `update_many` was not needed here

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-06-25 13:44:21 +00:00
697dfffe37 [ENG-670 / ENG-774] Resumable and Pausable jobs / Queued jobs don't show in the job manager (#961)
* Fixing some warnings

* Optimizing job workers

* Removing the delay from getting job reports

* remove some commented out stuff

* Trying to optimize job report route

* fix thread 'tokio-runtime-worker' panicked at 'attempt to subtract with overflow'

* fix progress bar

* Now pause works from the UI

* Fix

* Now the worker set job report to paused

* show errors for job pause/resume

* bunch 'o ui fixes

* order location in the sidebar

* fix some text

* fix clear all jobs

* fix clear jobs & job group ui

* show queued jobs

* object validator job text

* make cancel button work

* better executable logo

* Now cancel button works instantly

* disable resume button

* remove disabled props from pause/resume buttons

* remove large comments

* show paused progress

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-24 07:23:30 +00:00
Oscar BeaumontandJamie Pine b980a6f630 [ENG-792] App no starty (#1004)
* lul, message parsing with a response is cringe

* deadlocks hurt me physically

* super secret commit with many secret secrets

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-23 06:22:24 +00:00
187ec933f4 [ENG-770 / ENG-773] Copy/paste / Explorer context menu duplicate (#992)
* Fixing copy, cut and paste

* Bug on duplicating files without extensions

* Fix paste only ignoring the location origin when validating relative paths
 - Add Error handling to all context menu actions
 - Hide FS actions in Overview and MediaView
 - Remove redundant useExplorerSearchParams
 - Add default schema to the Zod param hooks

* Hide FS actions in all Explorers but location ones
 - Fix react warnings in RenameTextBox
 - Fix ReactTextBox not selecting the whole file name, instead of just up to file extension
 - Fix react warning due to inputing missing onChange

* basic show jobs for cut, copy and delete in job manager

* tweaks

* Fixing some warnings and minor tweaks

* Rust fmt

* Fix cut between locations on Linux

* Change how routes retrieve route params
 - Replace useZodRouteParams with useLoader
 - Define all schemas in the router file instead of in the route Component
 - Parse schema in loader to avoid requiring the route Components having to import the schema from the router file

* Remove default argument from useZodRouteParams

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: James Pine <ijamespine@me.com>
2023-06-22 23:41:29 +00:00
32824a88be [ENG-782] Seeded tags (#994)
* seeded tags infra

* dumb

* here's my seed

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-22 12:08:38 +00:00
jake f5d7a99a70 [ENG-737] Location deletion hanging (#978)
* fix concurrency issue with lib deletion

* clear only completed jobs

* slight fixes

* experimental but faster i think

* lower chunk size
2023-06-22 10:49:21 +00:00
Oscar Beaumont 7198775ea7 [ENG-783] rename node (#993)
rename node + hide some stuff
2023-06-22 08:35:09 +00:00
e8d3ad6005 [ENG-779] Finalize UI (#986)
* [ENG-779] Finalize UI
This is one branch with a variety of UI changes

add tag select mode bar without functionality 

fix group job status 

add notice icon with info to stat icons

add WIP notice to media view 

add modal before add location with greyed out clouds

remove disappearing add location button

add WIP spacedrop page 

bring back limited key manager UI 

add options bar on search page without functionality 

Add greyed out encrypt library button or setting

See more button on locations

Show locations on node screen

Fix overview category left padding

* key manager placeholder

* stat info

* nodes screen

* location click yay

* fix size in bytes

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>

* small ui improvements

* sh*tty see more button

* last touches

* fix merge boo boo

* Fix mobile
 - Move `getItemObject`, `getItemFilePath`, `getItemLocation`, `getExplorerItemData` to @sd/core to allow mobile to use them

* Formatting

* Normalize displayed file size between all screens
 - Replace every use of internal formatBytes with byte-size dep

---------

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-06-22 06:34:45 +00:00
Ericson "Fogo" SoaresandBrendan Allan de85f00efc [ENG-768 / ENG-769] Regen thumbnails / Generate checksums (#983)
* Making sub_path api for jobs harder to misuse

* re-enable context menu options

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-21 12:55:19 +00:00
Brendan Allan c6786f0c3f Better tag unassign (#989)
better tag unassign
2023-06-21 12:22:20 +00:00
jake aa57b153a7 Remove useless clone from mutation (#988)
* remove useless clone from mutation

* fix clippy too
2023-06-21 10:53:49 +00:00
jake ccaf8ad1c6 Fix tag removal (#985)
fix tag removal
2023-06-21 08:47:59 +00:00
Oscar BeaumontandBrendan Allan 37bf788ca5 [ENG-685] Sqlite Multithreaded (#917)
* pragma moment

* Try socket_timeout?

* bye

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 20:09:52 +00:00
Ericson "Fogo" SoaresandBrendan Allan 437f18081a [ENG-743 / ENG-744] Duplicate item / Delete explorer item (#977)
* Fixing a bunch of small bugs related to FS

* cleanup

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 09:44:13 +00:00
Brendan Allan 5df37d3a2d Do all category logic on server (#972)
* do all category logic on server

* cleanup
2023-06-19 12:21:32 +00:00
Brendan Allan fdda0dd280 Fix categories queries (#971)
* more stringent hidden filter check

* add missing migration case

* don't do sync stuff when sync disabled

* remove old stuff
2023-06-19 11:28:07 +00:00
Oscar Beaumont 43647479a9 ordering for create library is wrong (#970) 2023-06-19 09:01:56 +00:00
Oscar Beaumont 0f7a669e5a [ENG-732] Pairing by library (#943)
* library keypair

* allow opening stream to non-connected node + more

* library identity

* fix types

* fix maybe undefined

* don't forgor migrations file

* library manager inside p2p manager

* rename

* `NodeInformation` struct

* node info exchange

* fill in info

* streamify tunnel

* use tunnel for p2p events

* libp2p is annoying + stop leaking private key's

* Clippy cleanup
2023-06-19 05:13:30 +00:00
Brendan Allan 6975ebb664 Replace some default fields (#968)
replace some default fields
2023-06-19 00:48:38 +00:00
Brendan Allan d683d22c82 Remove owned sync (#967)
* remove owned sync + cleanup

* no more atomic records

* byebye owned

* remove owned ops from schema
2023-06-18 23:45:06 +00:00
Brendan Allan 2d703f2648 Make indexer rules optional (#965)
* optional indexer rules

* lockfile
2023-06-18 22:13:55 +00:00
Oscar Beaumont fd1afd8500 [ENG-568] Spacedrop Progress (#956)
* `Transfer` struct

* progress moment

* fixes
2023-06-18 08:40:51 +00:00