* Copy ephemeral files by chunks
* Improving buffered writer
* Report progress
* Copy and emit progress as two separated tasks
* Simplify for-if-return with `Iterator::any`
* Docs and use structured tracing
* Simplify code using paths
* wip
* wip
* wip
* wip
* Add report of messages while copying files
* Add info field to job report
* Group paths from OldFileCopierJobStep into a single struct
* Improve progress information
* Remove the need for synchronization
* Error handling for copy
* Clean up frontend
* Make watcher only consider new files
* Fix concurrent renaming of multiple files that have the same name
* Add documentation for file strategist
* Remove editions to ephemeral file copies
* Remove experimental/nightly features from Cargo.toml
* Fix no-case-declarations
* Remove dead comments
* Format code
* Use "Duplicate" instead of "Copy" in the frontend messages
* Remove inline always to make clippy happy
* icons for deleter and copier
* Fix JobManager for copy jobs
- Fix some types definitions
* Fix Job icon choosing logic
- Fix Copier job showing two progress bars
* Log which files are not formatted in the Type and style check CI
* fmt
* Forgot an import
* autoformat
---------
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* Migrate to Tauri v2 Release Canidate
* Formatting
* Update pnpm
* Update all tauri deps (js and rust) to Release Candidate 2
- A lot of misc fixes due to changes in the new versions of tauri, rspc and specta
* Recreate pnpm-lock
* Fix pnpm-lock
* Formatting
* Use 10 chars SHA for git deps
- Update a couple of rust dependencies
- Add taplo for toml auto formatting
- Adjust some formatting configurations
- Updated prettier and its plugins
* Fix formatter settings
* Minor improvements to Cargo.toml format
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* Done
* Updating storage statistics on volume watcher
* Don't sum up directories sizes for statistics
* storage bar ui changes
* changed storage bar colour gradient
* implemented loading state for statistics
* Fix migration
* Solving crashing and removing sleep
* Properly using useState I guess, not a TS dev
* fixed some ui issues
* Fixing reactivity issue
---------
Co-authored-by: myung03 <matthewyungisworking@gmail.com>
* Backend done
* Forgot new migration
* track subscription in file kind stats
* Real time updates on both graphs at overview
---------
Co-authored-by: myung03 <matthewyungisworking@gmail.com>
* initial ffprobe commit
* Working slim down version ffprobe
* Auto format ffprobe and deps source
* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning
* Remove show_* and print_format options and their respective logic
* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata
* Reduce the amount of ffmpeg log messages while generating thumbnails
* Fix completly wrong comments
* mend
* Start modeling ffmpeg extracted metadata on schema
- Start porting ffprobe code to rust
- Rename some references to media_data to exif_data
* Finish modeling media info data
- Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
- Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl
* Add relation between MediaInfo and FilePath
- Remove shared properties from MediaInfo and related structs
- Implement Iterator for FFmpegDict
* Fix and update schema
* Data models and start populating MediaInfo in rust
* Finish populating media info, chapters and program
* Improve FFmpegFormatContext data raw pointer access
- Implement stream data gathering
* Impl FFmpegCodecContext, retrieve codec information
- Improve some unsafe pointer uses
- Impl from FFmpegFormatContext to MediaInfo conversion
* Fix FFmpegDict Drop
* Fix some crago warnings
* Impl retrieval of video props
- Fix C char* to Rust String convertion
* Impl retrieval of audio and subtitle props
- Fill props for MediaCodec
* Remove simple_ffprobe now that the Rust impl is done
* Fix schema to match actually retrieved media info
- Fix import some FFmpeg constants instead of directly using values
* Rework movie_decoder
- Re-implement create_scale_string and add support anamorphic video
- Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
- Use newer FFmpeg abstractions in movie_decoder
* Fix incorrect props when initializing MovieDecoder
* Remove unecessary lifetimes
* Added more native wrappers for some FFmpeg native objects used in movie_decoder
* Remove FFmpegPacket
- Some more improvements to movie_decoder
* WIP
* Some small fixes
* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips
* fmt
* Fix duplicate migration for job error changes
* fix rebase
* Solving segfaults, fuck C lang
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Update rust to version 1.77
- Pin rust version with rust-toolchain.toml
- Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
- Remove unused function and imports
- Replace most CString uses with new c literal string
* More segfault solving and other minor fixes
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
- Plus some other misc fixes
* Auto format
* Retrieve video/audio metadata on frontend
* Auto format
* First draft on ffmpeg data save on db
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix some incorrect changes to prisma schema
* Some fixes for the FFmpegData schema
- Expand logic to save FFmpegData to db
* A ton of things
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Integrating ffmpeg media data in jobs and API
* Rspc can't BigInt
* 🙄
* Add initial ffmpeg metadata entries to Inspector
- Fix ephemeral metadata api to match the files metadata api call
* Fix Inspector not showing ffmpeg metadata
* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils
* Solving some issues
* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
- Fix humanizeSize breaking when receiving floating numbers
- Fix incorrect equality in StatCard
- Fix unhandled error in Dialog when trying to remove an unknown dialog
* fmt
* small improvements
- Remove some unecessary recursion_limit directive
- Remove unused app_image releated functions
- Fix metadata query enabled flag
* Add migration for ffmpeg media data
* Fix cypress test
* Requested changes
* Implement feedback
- Update locale keys for all languages
- Add pnpm command to update all language keys
* Fix thumb reactivity in non indexed locations
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Moving file-path-helper to a sub-crate on core
* Parallel walker tested and working
* Change inner core crate names to sd-core-* naming scheme
* Moving stuff around
* Save and Update tasks for the new indexer job
* Some initial drafts on the new job system
* More drafts on job system
* Changing walker task to a more explicit state machine
Also drafting more of job system
* More drafting on job resume
* Completed the draft on job system inner workings
* New job context abstraction to decouple library stuff from job system
* Properly use composition on task dispatcher
* First draft on indexer job
* Job serialization
* Handling ancestors in the distributed walker for indexer
* Saving computed directories sizes on a location to db
* Enable a way to check if jobs are running in a location
* Progress reporting on indexer job
* Reorganizing modules
* Shallow indexer
* Rust fmt
* Attempting windows CI fix
* Attempting windows CI fix again
* Attempting windows CI fix again
* Revert "Revert "remove fixed filters" (#2261)"
This reverts commit 5b40cefe37.
* search sources
* only set default filters if no filters
* key saved search page on id and use raw search as searchbar default
* fix crate versions
* put media view filters in a dedicated hook
* remove ts-reset
* add comment about <Inner>
* generics!
* cleanup
* search paths/objects switch (#2278)
* feature flag target switcher
* use useZodParams in saved search route
* P2P Debug route
* Remove legacy peer to peer pairing process
* Fix error typo
* Sync instances with cloud
* Upgrade deps + extended instance data
* Create instance with extended metadata
* Auto sync instances
* Actually `.await`
* bruh
* sync library info
* this isn't gonna work
* only sleep cloud receiver when no more messages (#1985)
* [ENG-1567] Fix renaming (#1986)
fix rename
* only sleep cloud receiver when no more messages
* use in memory instances during cloud receive (#1995)
* use in memory instances during cloud receive
* is_empty
---------
Co-authored-by: nikec <43032218+niikeec@users.noreply.github.com>
* fix type error
* wip
* make mdns mdns better
* rebuild state
* Add hooks + listeners + discovered state
* Split into crates
* wip fixing core + wip merging Spacetime into `sd-p2p2`
* `SmartLockGuard` + `Listener`
* Make `sd-core` compile
* Reenable all operation receivers
* Fix all broken code within `sd-core`
* minor fixes found in review
* Bring in `libp2p` + restructure `sd-p2p` for the gazillion-th time
* whoops
* Compile no matter the (runtime) cost
* fixing merge issues
* wip
* a
* b
* C
* Handle port betterer
* c
* Migrate node config
* a
* no crash on startup
* wip
* a
* jdfhskjfsg
* a
* fix discovery
* a bunch of fixes
* getting Spacedrop working
* I don't get why it no worky
* debug example
* a
* wip
* wip
* removing logging from stream impl
* wip: shit is fucked
* Redo quic integration + Spacedrop working
* Fix shutdown - deadlocks + shutdown peers
* Add Prisma migrations
* Fix shutdown
* a
* fix
* cleanup
* The lord clippy hath spoken
* disable P2P settings for now
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: nikec <43032218+niikeec@users.noreply.github.com>
* more sync support for file paths + saved searches
* sync support for labels
* update sync prisma generator to support more than tags
* workey
* don't do illegal db migration
* use name as label id in explorer
* First draft on image labeling
* Fixing execution providers for other OSs
* Better error handling and shutdown
* Working with shallow media processor
* bruh
* Fix warnings
* Now hooked to media processor job
* Link desktop app with libonnxruntime to avoid TLS error during startup
* Be able to change models on runtime
Revert to use labels table instead of tags
* A bug on a model-less inference
* Show AI labels on Inspector
- Change yolo inference to use half precision
- Add labels api to core
* Remove LD_PRELOAD
* Fix race condition on model executor shutdown
* Don't load all images in memory moron
* Embeed yolo model in prod build
- Change yolo model path to new one relative to executable
* Disable volume watcher on linux, it was crashing the app
- Invalidate labels when they are updated
* Rust fmt
* Minor changes
* Gate onnxruntime linking to the ai-models feature
* Add build script to sd-server to handle onnxruntime linking workaround
* Move AI stuff to its own crate and normalize deps
* Rust fmt
* Don't regenerate labels unless asked to
* Now blazingly fast
* Bad merge
* Fix
* Fix
* Add backend logic to download extra yolo models
* Add models api route
- Add api call to get available model version
- Add api call to change the model version
* Improve new model download logic
- Add frontend to change image labeler model
* Fix new model downloader
* Fix model select width
* invalidate labels count after media_processor generates a new output
* Rename AI crate and first draft on download notifications
* fix types
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* search options start
* small progress
* more
* bunch of stuff
* semi functioning filters
* cleanup setup api
* progress
* remove filters
* hooked up to query epic moment
* fix
* move db stuff to specific modules
* in/notIn for some fields
* generate ts
* big gains
* working filter options for locations, tags and kind
* working search query
* perfect fixed filters
* saved searches lol
* merge error
* saved searches via api
* better routing
* [ENG-1338] Fix fresh Spacedrive install failing to start due to attempting to query a nonexistent Library (#1649)
Fix Spacedrive failing to start due to attempting to query a nonexistent Library
- Rename useShoudRedirect to useRedirectToNewLocations
- Improve behaviour for the immedite redirection after adding a new location
* Show hidden files false by default (#1652)
bool
* fix remove filter in list
* tweaks
* fix nav buttons
* unify MediaData search handling
* cleanup saved search writing
* Add left top bar portals for tags and search + fixed media view on tags
* added search to filter dropdown
* render cycle improvements
* hotfix
* wip
* Refactor with Brendan, but this is a WIP and the search query no longer works
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
* progress
* fix location/$id page
* fix tags too
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
* 3rd refactor lol
epic style
* half-done with enum-ification of SearchFilterArgs
* broken fixed filters but working inNotIn filters
* search name + extension kinda working
* hidden filter
* fixed filters working??
* deferred search value
* extensions works
* filtered search items mostly working
* tweaks
* stacked approach working for non-search filters
* move to Explorer/Search
* actually use filterArgs in queries
things actually work properly now
* added new icons from Mint
* goof
* cleanup types, filters and mutation logic
* actually use search value
* remove overview from sidebar
* don't shrink LibrariesDropdown ga
* remove overview from sidebar and default to /network
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
* clippy -_-
* migrations
* alter the migration so it just renames the `dimensions` field (no db re-creation required)
* remove confusing semver addition for `libheif-sys`
* remove warning on the migration as it's just a rename
* add sort opts for resolution and date image was truly taken
* major serde ckeanup & add epoch_time and pixel_count
* rename, cleanup et optimise
* clippy
* ignore this mess
* bindings
* add explanation to schema
* comment out dt test
* better comment and WIP time
* cleanup rust
* failed timezone attempt
* remove image resolution as a sort by option
* update schema (and rename the dimensions table instead of dropping it)
* just show raw date
* add comments and update bindings
* fix migration hopefully
* fix broken migration
---------
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* basic layout
* lock
* add codec to image
* very messy wip
* rm that
* lock and toml
* working perfect exif extraction
* formatting
* migration and formatting
* mostly working
* fix
* set date created properly
* fix tsc
* working media data creation
* fix bad main merge? sorry brendan
* schema, migrations, bindings
* working exif data extraction
* why won't it work
* update migrations
* fix bad merge
* final cleanup
* cleanup migrations
* remove test (that was purely used for testing my code)
* working media data pulling, correct thumbnail orientation
* slightly optimise thumbnail rotation
* rename location to prevent specta clashes
* further improvements (location parsing is still broken)
* fix coordinate parsing i think
* rspc add some todos regarding final cleanup
* further thoughts
* major upgrades
* Some improved handling of errors and async calls
* accordion component
* heavily wip flash refactor
* fix builds all because of a (terrible) merge/move
* annoying missing newline
* i really hate exif
* remove dead code
* further flash progress :D
* docs(media-data): clarification
* minor cleanup
* cleanup and some async removal
* fix location parsing
* remove async (will do proper impl for async eventually) and major cleanup
* big W
* clippy and `FlashMode::Unknown` instead of `Invalid`
* add `NIKON` gps ref support
* comments about gps refs
* commit the submodule -_-
* major cleanup & composite image support
* remove old test image
* major cleanup and easier to use API
* remove old consts
* move `ExifReader` to dedicated module
* Media Data Extractor job and shallow job
* Extracting media data on watcher
* report no exif data on file gracefully
* cleanup errors and doctests
* Merging thumbnailer and media data extractor
* Job name and some strings in UI
* remove reliance on `sd-prisma` within the media data crate
* rename query to be more specific
* custom serializer for `MediaTime`
* tweak to format utc -> naive correctly
* generate migrations
* comment out duration in mobile
* delete test-assets folder
* all optional fields
* fix migrations again
* make requested name changes
* make further requested changes
* remove erroneous files from another wip branch
* updates procedures
* use strings where appropriate
* regen pnpm-lock
* add base layouts for video and audio media data
* use appropriate data types in schema and add audio media data framework
* make requested changes
* general cleanup and renaming of enum
* cleanup media data api
* rename media metadata type
* finishing touches
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* less stupid name
* yeet
* awaiting futures is kinda important lol
* no-async
* more proto stuff
* cleanup
* move it round
* None of my homies like broadcast
* minor
* do the shuffle
* restore after force push
* reusing `sysinfo::System` as intended
* fix lol
* remove `node_id` from `Volume`
* Remove `node_local_id` from `Library`
* Remove `Job` to `Node` relation
* feature flags be like
* press save 4head
* remove `Location` -> `Node` relation
* `volume.rs` to `volume/mod.rs`
* yes
* add `Instance` model and deprecate `Node` model
* pairing is better now
* Pretty code
* thinking in code
* wip
* What if Observables but in Rust?!
* Observables aren't it + `useP2PEvents` hook
* more around some jsx
* Trade offer: bad code for working pairing?
* Unit test pairing protocol
* fix everything up
* corrections
* Have you got a moment for our lord and saviour Clippy
* tsc --fixMyCode
* Prisma being wacky
* Config files being f'ed up
* broken config after migrating
* Zed being Zed
* Argh
* cliipzy
* rewind
* Fix truncate logic
* wip: instances in peer metadata
* Rethink instance ids
* fix
* whoops
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
