59 Commits

Author SHA1 Message Date
renovate[bot]
239fddf39c Update all dependencies (develop) (#3238)
* Update all dependencies

* Compare fakefs result after sorting

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Safihre <safihre@sabnzbd.org>
2025-12-22 12:45:00 +00:00
Safihre
a91e718ef5 Split nzbstuff into separate files for Article, NzbFile and NzbObject (#3221) 2025-12-09 21:21:51 +01:00
jcfp
e2f613da6c stop catching BaseException (#3062) 2025-04-09 15:27:22 +02:00
thezoggy
dd0122865a bump copyright year (#3034) 2025-02-17 14:06:12 +01:00
Safihre
7c607a6bf6 Simplify file and folder name sanitization 2024-12-30 22:15:14 +01:00
Safihre
93c6d67ef8 Improve handling of too long utf8 file and folder names
Closes #2991
Closes #2988
2024-12-30 14:59:26 +01:00
jcfp
1519dbc554 Remove pyfakefs workarounds and put its new apply_umask option to good use (#2922) 2024-08-16 15:44:52 +02:00
jcfp
08b1b20b34 fix filesystem permission test for pyfakefs 5.4.0+ (#2905) 2024-07-10 16:16:18 +02:00
Safihre
567608b3c4 Using password=PW resulted in PW.nzb as password
Closes #2779
2024-01-17 14:51:35 +01:00
thezoggy
9f6a9f9912 normalize and update copyright year 2024-01-02 09:42:17 +01:00
Safihre
0dddaf26e0 Stricter validation on Windows to prevent network drives as Incomplete 2023-12-02 21:19:48 +01:00
Safihre
b5fbc8af86 Refactor handling of Complete vs Incomplete check
Closes #2717
2023-10-25 16:06:28 +02:00
Safihre
7cbbff727f Remove (almost) all references to unused team@sabnzbd.org 2023-07-21 11:33:51 +02:00
Sander
e65980258c test writing long and unicode filenames (#2542)
* test writing long and unicode filenames

* Update sabnzbd/filesystem.py

Co-authored-by: Safihre <safihre@sabnzbd.org>

* Update sabnzbd/filesystem.py

Co-authored-by: Safihre <safihre@sabnzbd.org>

* Update sabnzbd/filesystem.py

Co-authored-by: Safihre <safihre@sabnzbd.org>

* Update sabnzbd/filesystem.py

Co-authored-by: Safihre <safihre@sabnzbd.org>

* test writing long and unicode filenames

* seperate function test_filesystem_capabilities

* rename test_filesystem_capabilities to filesystem_capabilities

* rename filesystem_capabilities to check_filesystem_capabilities

---------

Co-authored-by: sander <san.d.erjonkers+github@gmail.com>
Co-authored-by: Safihre <safihre@sabnzbd.org>
2023-05-08 11:50:49 +02:00
Thomas
dcca2c5821 Check for errors when creating the download path (#2516)
Fixes https://github.com/sabnzbd/sabnzbd/issues/2515.
2023-03-28 17:54:39 +02:00
Safihre
98f3c055d7 Ignore resource fork files created by macOS
Closes #2380
2023-03-19 21:41:31 +01:00
puzzledsab
6cd5713baa Translate ascii control chars below value 32 to _ (#2463)
* Translate ascii control chars below value 32

* Try to make code and tests consistent

* More test fixing

* Delete too much

* Different approach

* Finally got it?

* Start from 0

* Convert \0 to _ for all systems

* Check if CH_ILLEGAL_WIN is translated to CH_LEGAL_WIN

* Test specific chars
2023-02-18 22:48:00 +01:00
Safihre
fe0c4e4f92 Update formatting with black 23 rules 2023-02-01 20:42:06 +01:00
Safihre
6492cfb430 Update copyright year to 2023 2023-01-25 22:39:49 +01:00
thezoggy
cc402e35a1 codespell (#2321)
* ran through codespell to cleanup a bit of typos, excluded a bit to try and avoid any issues

* fix typos - may need review
2022-10-31 22:51:38 +01:00
jcfp
ee4b9339a7 use OSType in pyfakefs instead of setting separate properties (#2243) 2022-07-19 22:36:44 +02:00
Sander
df8859d49e Check filesystem capability for writing and writing filenames with special characters (#2167)
* functions to test directory for writing capabilities

* functions to test directory for writing capabilities

* use checking in postproc, and give warning if needed

* use checking in postproc, and give warning if needed

* put into function, with translatable folder names

* remove test file if still there

* better message formatting

* remove friendly directory name. Less comments

* move stuff into filesystem.py

* clean it up

* unit test for check_directory_writing_capability on tempdir

* unit test for check_directory_writing_capability on tempdir

* unit test for check_directory_writing_capability on tempdir

* unit test for check_directory_writing_capability on tempdir

* unit test for check_directory_writing_capability on tempdir

* Update sabnzbd/filesystem.py

Co-authored-by: Safihre <safihre@sabnzbd.org>

* feedback processed

* feedback processed

* feedback processed

* Merge remote-tracking branch 'origin/check_filesystem_capabilty' into check_filesystem_capabilty

# Conflicts:
#	sabnzbd/filesystem.py

* typo: uniformed on "writable"

Co-authored-by: Safihre <safihre@sabnzbd.org>
2022-05-11 19:40:19 +02:00
Safihre
6c6c8c86d8 Rename DARWIN constant to MACOS 2022-01-25 13:44:51 +01:00
Safihre
9c7b61070b Update copyright year to 2022 2022-01-07 09:29:32 +01:00
Safihre
0bb5432223 Do not apply permissions if not requested by the user, but remove xbits (#2004)
* Do not apply permissions if not requested by the user, but remove xbits

* Verify umask and user-permissions and warn for potential access problems

* Correctly name umask/permissions options

* Apply permissions only on download_dir creation

* Refactor some permissions related actions

* Block setting permissions=0 and only stat when no custom_permissions
2022-01-03 10:43:57 +01:00
jcfp
255242eca5 disable randomisation of directory listings in pyfakefs (#1978) 2021-11-18 15:26:26 +01:00
jcfp
7dbde008af Regex for unwanted extensions (#1907)
* allow regexp for unwanted extensions

* housekeeping

* use rss.convert_filter

* improve function name

* move convert_filter to misc
2021-06-01 07:28:19 +02:00
jcfp
9fb03a25f6 allow missing extension for unwanted check (#1896) 2021-05-16 21:02:06 +02:00
Safihre
cdd7e6931a Post-processing would crash if there is no files to unpack 2021-05-01 16:50:15 +02:00
Safihre
8ffa3e5d4c Add unit tests for sanitize_files 2021-04-27 22:51:43 +02:00
Safihre
615c296023 sanitize_files_in_folder would ignore the newfiles
Would result in deobfuscate not working.
This needs unittests!
Closes #1868
2021-04-27 12:09:46 +02:00
Safihre
c93de2dd6f Correct set_config fixture in unittests 2021-04-22 21:36:13 +02:00
jcfp
d491eb1af0 fix use of dir (#1855) 2021-04-16 16:25:20 +02:00
Safihre
8a39e5827b Foldernames should always be stripped from dots and spaces at the end 2021-04-01 07:28:08 +02:00
Sander
0a8747f600 pre-create subdir if needed (POSIX, par2) (#1802)
* pre-create subdir it needed

* pre-create subdir it needed: check if already exists

* use os.makedirs() to handle subdir1/subdir2/blabla

* protect against malicous "..", and better naming

* check for Windows \ and POSIX /

* check again within path, typo and formatting

* regex: square brackets

* cleanup: only "/" can occur in par2

* cleanup: better logging

* unit test: testing of filesystem.renamer()

* if subdir specified in par2: let filesystem.renamer() do all the work

* if create_local_directories=True, then renamer() must stay within specified directory. Plus unittest for that.

* if create_local_directories=True, then renamer() must stay within specified directory. Plus unittest for that.

* more comments in code

* use filesystem.create_all_dirs(), less logging, clearer "..", and other feedback from Safihre

* make remote black happy too

* Small changes in wording of comments and error

Co-authored-by: Safihre <safihre@sabnzbd.org>
2021-02-28 11:42:49 +01:00
jcfp
24d3d064bb add unwanted extensions whitelist mode (#1798)
* add unwanted extensions whitelist mode

* only call get_ext once

* remove unneeded .lower()
2021-02-14 17:05:26 +01:00
Sander
6d40eba496 Filename limit in sanitize_filename() to avoid traceback (#1721)
* urlgrabber limit filename to avoid tracebacks

* urlgrabber limit filename to avoid tracebacks: black

* urlgrabber limit filename to avoid tracebacks: black

* filename_limit ... 2020-01-15

* filename_limit: into sanitize_filename()

* filename_limit: black and typo and logging

* filename_limit: debug show full filename

* filename_limit: unittests

* sanitize_filename(): take care of feedback: one ASCII method, handly silly extension lengths

* sanitize_filename(): tests/test_filesystem.py ... make black happy

* sanitize_filename(): typo in comment

* sanitize_filename(): test_filesystem.py ... black

* sanitize_filename(): more unittests, and DEF_FILE_MAX (yet without GUI option)

* sanitize_filename(): always use DEF_FILE_MAX

* sanitize_filename(): black

* sanitize_filename(): handle UTF8 correct (>1 byte). DEF_FILE_MAX = 255

* sanitize_filename: measure bytes (not chars), DEF_FILE_MAX = 255 - 6, no test-writing in unittests

* sanitize_filename: constants.py ... black

* sanitize_filename: comment about extension

* DEF_FILE_MAX = 255 - 10 again, to solve adding ".nzb.gz" elsewhere
2021-01-19 22:03:13 +01:00
Safihre
f0bfb08c2e Update copyright year to 2021 2021-01-03 20:43:19 +01:00
Safihre
b75c23772e Remove unused function is_obfuscated_filename 2020-12-23 11:38:50 +01:00
Safihre
93b429af8b We do not need to trim incomplete paths on Windows anymore 2020-09-27 17:57:10 +02:00
Safihre
0108730004 Set a maximum on the maximum length of a foldername
Closes #1597
2020-09-06 16:40:39 +02:00
Safihre
a92d2b585e Permissions would be set even if user didn't set any
Windows developers like me shouldn't do permissions stuff..
2020-08-21 15:04:46 +02:00
Safihre
df19d4d323 Extend tests of create_all_dirs to cover apply_umask=False 2020-08-18 13:59:39 +02:00
Safihre
bcf6a5bd09 Permissions were not set correctly when creating directories (#1568)
Restores changes made in d2e0ebe
2020-08-07 15:22:27 +02:00
Safihre
4420c36889 Speed-up selenium tests by re-using open browser window 2020-06-18 14:18:11 +02:00
Safihre
92efc9ed8a Improve automated download-testing
- Catch all Selenium exceptions while waiting for the download to complete
- Correctly set autodisconnect=0
- Make job-name even more unique
- Wait for the RSS-added job to finish fetching
- Move sabnzbd.basic.ini to data folder
- Use new SAB_DATA_DIR
- Optimized imports
2020-06-17 22:33:21 +02:00
Safihre
7e5a43e95c Revert "Code-formatting: optimize imports"
This reverts commit db5ff8c7e1.

Does not work on Python 3.4/3.6. Not really sure why, but don't want to investigate right now.
2020-06-14 11:29:54 +02:00
Safihre
db5ff8c7e1 Code-formatting: optimize imports 2020-06-14 10:21:33 +02:00
Safihre
8ce6c26f9c Do not scan incomplete directory recursively
Also renamed the recursive_listdir function so it can also be non-recursive. Which was very easy due to the extensive test-set by @jcfp!
Relates to #1304 (that NZB triggered invalid reading of sub-dirs in incomplete folder)
2020-06-13 21:36:48 +02:00
jcfp
e9b4340a25 extend filesystem tests (#1481)
* extend filesystem tests

* fix test failure when no explicit umask was set

* have black uglify the code

* require case-sensitive fs for test_capitalization_linux

* run black with -l120 instead

* make windows-compatible, fix some minor issues

* mark xfail rather than comment out part of trim_win_path
2020-06-03 15:57:52 +02:00