Commit Graph

401 Commits

Author SHA1 Message Date
Ericson "Fogo" Soares
26e3b53b21 Small fix due to how sharding works (#1138) 2023-07-28 03:46:44 +00:00
Ericson "Fogo" Soares
aec0c94d62 [ENG-794] Library deletion hangs (#1137)
Solving deadlock on library deletion
Also improving some minor stuff on thumbnails remover actor
2023-07-27 23:18:45 +00:00
jake
7215e40a3b [ENG-926] Prevent thumbnail destruction and fix the remover (#1127)
* fix(core): thumbnail removal

* chore(core): add todo

* New actor on steroids

* Improving thumbnail remover actor

* Ignoring errors from files that doesn't exist

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-07-25 14:21:50 +00:00
Ericson "Fogo" Soares
9283e74d85 [ENG-875] Re-index & Re-thumb should be smart (#1130)
* New sub_path rescan rspc route

* Changing some rescan call sites to use new route
2023-07-25 12:47:56 +00:00
Brendan Allan
447069388b Push libraries into vec after loading (#1129)
push libraries into vec after loading
2023-07-24 17:17:02 +00:00
Brendan Allan
043b607ad4 CRDTOperation receiving (#1122)
* operation receive + compare

* cleanup + deduplication

* operation receive + compare

* cleanup + deduplication

* sync route + operation grouping

* tag assign sync

* proper relation support in sync debug page

* migration

* separate core-sync + utils crates

* separate p2p event loop from manager

* cleanup library handling

* clippy

* feature gate sync messages properly

* make migration not add required field
2023-07-24 15:26:00 +00:00
Oscar Beaumont
23b0c403be [ENG-925] - Arc<Library>/Arc<NodeContext> (#1124)
* `Arc<Library>` + `Arc<NodeContext>`

* fix specta version

* remove unneccessary derefs

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 10:59:57 +00:00
Ericson "Fogo" Soares
03e71e98a4 [ENG-778] Location clean up job (#1123)
* Introducing thumbnail remover actor
Also tweaking the orphan remover actor

* Rust fmt
2023-07-20 15:17:54 +00:00
Oscar Beaumont
53ab3178c2 [ENG-906] Initial library sync (#1095)
* ffs

* typo

* yeet library data over p2p

* fix a bunch of edge cases

* report complete status on responder

* better log

* fix types

* mobile debug screen

* mobile + P2P is a mess

* feature flag mobile p2p pairing

* wrong one

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-17 11:53:25 +00:00
Brendan Allan
0a1d721e58 Use codegen for ModelSyncType exec (#1098)
* use codegen for ModelSyncType exec

* folder client format

* hande create operation properly
2023-07-15 04:22:04 +00:00
Brendan Allan
f9b32f5a4a SQLite connection limit (#1111)
connection_limit=1
2023-07-15 03:56:00 +00:00
Oscar Beaumont
1a5d2898c2 [ENG-537] Notification system (#805)
* `Notification` struct

* Manage dem notifications

* a dash of rspc

* `read` field

* `handle_notification_callback`

* some Prisma

* TS-centric notifications backend

* `useNotifications` hook

* Notification test buttons
2023-07-12 07:43:27 +00:00
Oscar Beaumont
cf39f8dbcc [ENG-759] P2P Cleanup (#1062)
* less stupid name

* yeet

* awaiting futures is kinda important lol

* no-async

* more proto stuff

* cleanup

* move it round

* None of my homies like broadcast

* minor

* do the shuffle

* restore after force push

* reusing `sysinfo::System` as intended

* fix lol

* remove `node_id` from `Volume`

* Remove `node_local_id` from `Library`

* Remove `Job` to `Node` relation

* feature flags be like

* press save 4head

* remove `Location` -> `Node` relation

* `volume.rs` to `volume/mod.rs`

* yes

* add `Instance` model and deprecate `Node` model

* pairing is better now

* Pretty code

* thinking in code

* wip

* What if Observables but in Rust?!

* Observables aren't it + `useP2PEvents` hook

* more around some jsx

* Trade offer: bad code for working pairing?

* Unit test pairing protocol

* fix everything up

* corrections

* Have you got a moment for our lord and saviour Clippy

* tsc --fixMyCode

* Prisma being wacky

* Config files being f'ed up

* broken config after migrating

* Zed being Zed

* Argh

* cliipzy

* rewind

* Fix truncate logic

* wip: instances in peer metadata

* Rethink instance ids

* fix

* whoops

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:23:30 +00:00
Niklas Wojtkowiak
c877c03b63 Add functionality to some Overview Catrgories (#1090)
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:01:30 +00:00
Brendan Allan
07c00a8b7f [ENG-846] Preferences (#1047)
* preferences table + trait

* cleaner implementation

* router + ts type

* preference test

* per-column list view preferences

* preference read + write

* add some docs

* migration + docs

* remove ts expect error
2023-07-12 04:32:53 +00:00
Ericson "Fogo" Soares
7cd00dab49 [ENG-884] Update file_path metadata (#1077)
* Partitioned paths to be updated on walker

* Updating file_paths in indexer

* Properly tracking modified date on renames

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-11 15:14:11 +00:00
nikec
9d800eec31 [ENG-885] Move online locations to lib store (#1074)
* Enable trace logging for invalidate query

* Fixing quick rescan on explorer
And annotating wrong quick rescan usage on top bar reload button

* Move online locations to lib store

* Fix topbar reload path

* fix mobile

* Remove unused imports + format
 - Adjust `arraysEqual` arguments type

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-05 22:27:38 +00:00
Vítor Vasconcellos
f83f29c720 [ENG-765] Reported Total capacity and Free space are wrong (#1066)
* Attempt at fixing stats

* Fix macOS disk stats retrieve logic
 - Ignore mounted dmgs when calculation disk total/free size
 - Only take into account disk mounted by macOS

* macos only import

* Fix Linux

* Replace byte-size with a custom implementation that supports BigInt

* Fix NaN in Statistics

* clippy

* fmt

* Move linux get_volumes to a specilized function
 - Fix ZFS handling
 - Improve handling of disk symlinks and multiple mounts

* Fix macOS
2023-07-05 18:22:56 +00:00
Matthias Berchtold
52c5d1d2e9 Indexer rule No Git (#1048)
feat: no git indexer

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-07-03 11:45:36 +00:00
Brendan Allan
f1239d166a Search filter cleanup (#1063)
cheeky cleanup
2023-07-03 03:02:57 +00:00
Ericson "Fogo" Soares
dcd071d229 [ENG-850] Wrongly labelled .ts files (as video) (#1052)
* Fixing conflits of mts code files showing as video

* Introducing a reidentify_objects param on fullRescan

* Fix job output metadata in case of completed with errors
2023-06-29 02:58:33 +00:00
Ericson "Fogo" Soares
5d49c8a74a [ENG-823] Fixing bad handling of dots in directory name (#1051)
* Fixing bad handling of dots in directory name

* fix ci not running on rust

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-28 21:41:55 +00:00
Ericson "Fogo" Soares
1dce6380fe Job System and Watcher improvements (#1030)
* Trying to make watcher more resilient between creates and updates

* More checks on create vs update

* Rust fmt

* PCR rev

* Windows CI fix

* Some indexer improvements

* Builder pattern for jobs and reports

* fix list view invalid date

* fix progress bar

* Normalizing job metadata output

* fixed job metadata

* Removing inner report struct from identifier job

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-06-28 16:30:19 +00:00
Brendan Allan
444a6d23c0 [ENG-806] Better library name handling (#1043)
better library name handling
2023-06-28 09:11:24 +00:00
Utku
bc100f2505 Fix job resuming/pausing and cancelling (#1035)
why this works?
2023-06-27 19:26:59 +00:00
Oscar Beaumont
d5ddfcc667 Job System - Simplify data (#1017)
* yeet

* destructure `JobState` when passed to `finalize`.
Let him cook.

* `impl Job for JobInitData` instead of it being `Job::Init`
The pan is on high heat and the 3 sh-meats are coming out.

* Remove `JobInitData` & `IntoJob` & `Job::new`

* clean up a little

* Remove `Job.stateful_job` in favor of `Job.state.init`
Yeah this isn't great but lets get these changes working and then think again about it.

* Properly call `StatefulJob.finalize`

* she'll be right

* fix `initialize_resumable_job`

* builder syntax for creating jobs

* Have Rust gone too far?

* Minor fixes

* fixes

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-06-27 07:15:49 +00:00
Oscar Beaumont
f86b6c8b52 [ENG-823] Fix some stuff (#1024)
* SQLite journal bug

* Remove "job.report" indexes thing

* resume button - job not found?

* panic hook with logging

* it did

* large pain

* fix `check_nested_location`

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-27 05:12:04 +00:00
Oscar Beaumont
ac5f6323a5 Job System - Better Progress (#1015)
yeet

Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-06-26 17:49:55 +00:00
Gedeon
5a790ac996 [ENG-803] Set date upon tag creation (#1021)
* set date_created to tag

* update tag date modified when edited

* use `FIxedOffset` for tag `date_created`

* remove dead code and use fixedoffset

* discard duplicate code

* remove dead code & add current time to field

* remove unused code

* no longer needed in data

* set creation time for tags

* fix tag creation time

* fix & finalize timestamps

* `update_many` was not needed here

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-06-25 13:44:21 +00:00
Ericson "Fogo" Soares
697dfffe37 [ENG-670 / ENG-774] Resumable and Pausable jobs / Queued jobs don't show in the job manager (#961)
* Fixing some warnings

* Optimizing job workers

* Removing the delay from getting job reports

* remove some commented out stuff

* Trying to optimize job report route

* fix thread 'tokio-runtime-worker' panicked at 'attempt to subtract with overflow'

* fix progress bar

* Now pause works from the UI

* Fix

* Now the worker set job report to paused

* show errors for job pause/resume

* bunch 'o ui fixes

* order location in the sidebar

* fix some text

* fix clear all jobs

* fix clear jobs & job group ui

* show queued jobs

* object validator job text

* make cancel button work

* better executable logo

* Now cancel button works instantly

* disable resume button

* remove disabled props from pause/resume buttons

* remove large comments

* show paused progress

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-24 07:23:30 +00:00
Oscar Beaumont
b980a6f630 [ENG-792] App no starty (#1004)
* lul, message parsing with a response is cringe

* deadlocks hurt me physically

* super secret commit with many secret secrets

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-23 06:22:24 +00:00
Ericson "Fogo" Soares
187ec933f4 [ENG-770 / ENG-773] Copy/paste / Explorer context menu duplicate (#992)
* Fixing copy, cut and paste

* Bug on duplicating files without extensions

* Fix paste only ignoring the location origin when validating relative paths
 - Add Error handling to all context menu actions
 - Hide FS actions in Overview and MediaView
 - Remove redundant useExplorerSearchParams
 - Add default schema to the Zod param hooks

* Hide FS actions in all Explorers but location ones
 - Fix react warnings in RenameTextBox
 - Fix ReactTextBox not selecting the whole file name, instead of just up to file extension
 - Fix react warning due to inputing missing onChange

* basic show jobs for cut, copy and delete in job manager

* tweaks

* Fixing some warnings and minor tweaks

* Rust fmt

* Fix cut between locations on Linux

* Change how routes retrieve route params
 - Replace useZodRouteParams with useLoader
 - Define all schemas in the router file instead of in the route Component
 - Parse schema in loader to avoid requiring the route Components having to import the schema from the router file

* Remove default argument from useZodRouteParams

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: James Pine <ijamespine@me.com>
2023-06-22 23:41:29 +00:00
Brendan Allan
32824a88be [ENG-782] Seeded tags (#994)
* seeded tags infra

* dumb

* here's my seed

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-22 12:08:38 +00:00
jake
f5d7a99a70 [ENG-737] Location deletion hanging (#978)
* fix concurrency issue with lib deletion

* clear only completed jobs

* slight fixes

* experimental but faster i think

* lower chunk size
2023-06-22 10:49:21 +00:00
Oscar Beaumont
7198775ea7 [ENG-783] rename node (#993)
rename node + hide some stuff
2023-06-22 08:35:09 +00:00
Jamie Pine
e8d3ad6005 [ENG-779] Finalize UI (#986)
* [ENG-779] Finalize UI
This is one branch with a variety of UI changes

add tag select mode bar without functionality 

fix group job status 

add notice icon with info to stat icons

add WIP notice to media view 

add modal before add location with greyed out clouds

remove disappearing add location button

add WIP spacedrop page 

bring back limited key manager UI 

add options bar on search page without functionality 

Add greyed out encrypt library button or setting

See more button on locations

Show locations on node screen

Fix overview category left padding

* key manager placeholder

* stat info

* nodes screen

* location click yay

* fix size in bytes

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>

* small ui improvements

* sh*tty see more button

* last touches

* fix merge boo boo

* Fix mobile
 - Move `getItemObject`, `getItemFilePath`, `getItemLocation`, `getExplorerItemData` to @sd/core to allow mobile to use them

* Formatting

* Normalize displayed file size between all screens
 - Replace every use of internal formatBytes with byte-size dep

---------

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-06-22 06:34:45 +00:00
Ericson "Fogo" Soares
de85f00efc [ENG-768 / ENG-769] Regen thumbnails / Generate checksums (#983)
* Making sub_path api for jobs harder to misuse

* re-enable context menu options

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-21 12:55:19 +00:00
Brendan Allan
c6786f0c3f Better tag unassign (#989)
better tag unassign
2023-06-21 12:22:20 +00:00
jake
aa57b153a7 Remove useless clone from mutation (#988)
* remove useless clone from mutation

* fix clippy too
2023-06-21 10:53:49 +00:00
jake
ccaf8ad1c6 Fix tag removal (#985)
fix tag removal
2023-06-21 08:47:59 +00:00
Oscar Beaumont
37bf788ca5 [ENG-685] Sqlite Multithreaded (#917)
* pragma moment

* Try socket_timeout?

* bye

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 20:09:52 +00:00
Ericson "Fogo" Soares
437f18081a [ENG-743 / ENG-744] Duplicate item / Delete explorer item (#977)
* Fixing a bunch of small bugs related to FS

* cleanup

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 09:44:13 +00:00
Brendan Allan
5df37d3a2d Do all category logic on server (#972)
* do all category logic on server

* cleanup
2023-06-19 12:21:32 +00:00
Brendan Allan
fdda0dd280 Fix categories queries (#971)
* more stringent hidden filter check

* add missing migration case

* don't do sync stuff when sync disabled

* remove old stuff
2023-06-19 11:28:07 +00:00
Oscar Beaumont
43647479a9 ordering for create library is wrong (#970) 2023-06-19 09:01:56 +00:00
Oscar Beaumont
0f7a669e5a [ENG-732] Pairing by library (#943)
* library keypair

* allow opening stream to non-connected node + more

* library identity

* fix types

* fix maybe undefined

* don't forgor migrations file

* library manager inside p2p manager

* rename

* `NodeInformation` struct

* node info exchange

* fill in info

* streamify tunnel

* use tunnel for p2p events

* libp2p is annoying + stop leaking private key's

* Clippy cleanup
2023-06-19 05:13:30 +00:00
Brendan Allan
6975ebb664 Replace some default fields (#968)
replace some default fields
2023-06-19 00:48:38 +00:00
Brendan Allan
d683d22c82 Remove owned sync (#967)
* remove owned sync + cleanup

* no more atomic records

* byebye owned

* remove owned ops from schema
2023-06-18 23:45:06 +00:00
Brendan Allan
2d703f2648 Make indexer rules optional (#965)
* optional indexer rules

* lockfile
2023-06-18 22:13:55 +00:00
Oscar Beaumont
fd1afd8500 [ENG-568] Spacedrop Progress (#956)
* `Transfer` struct

* progress moment

* fixes
2023-06-18 08:40:51 +00:00