Add X509 certificate chain based signing with PKCS7-PEM external
signatures distributed alongside CVD's in a custom .cvd.sign format.
This new signing and verification mechanism is primarily in support
of FIPS compliance.
Fixes: https://github.com/Cisco-Talos/clamav/issues/564
Add a Rust implementation for parsing, verifying, and unpacking CVD
files.
Now installs a 'certs' directory in the app config directory
(e.g. <prefix>/etc/certs). The install location is configurable.
The CMake option to configure the CVD certs directory is:
`-D CVD_CERTS_DIRECTORY=PATH`
New options to set an alternative CVD certs directory:
- Commandline for freshclam, clamd, clamscan, and sigtool is:
`--cvdcertsdir PATH`
- Env variable for freshclam, clamd, clamscan, and sigtool is:
`CVD_CERTS_DIR`
- Config option for freshclam and clamd is:
`CVDCertsDirectory PATH`
Sigtool:
- Add sign/verify commands.
- Also verify CDIFF external digital signatures when applying CDIFFs.
- Place commonly used commands at the top of --help string.
- Fix up manpage.
Freshclam:
- Will try to download .sign files to verify CVDs and CDIFFs.
- Fix an issue where making a CLD would only include the CFG file for
daily and not if patching any other database.
libclamav.so:
- Bump version to 13:0:1 (aka 12.1.0).
- Also remove libclamav.map versioning.
Resolves: https://github.com/Cisco-Talos/clamav/issues/1304
- Add two new API's to the public clamav.h header:
```c
extern cl_error_t cl_cvdverify_ex(const char *file,
const char *certs_directory);
extern cl_error_t cl_cvdunpack_ex(const char *file,
const char *dir,
bool dont_verify,
const char *certs_directory);
```
The original `cl_cvdverify` and `cl_cvdunpack` are deprecated.
- Add `cl_engine_field` enum option `CL_ENGINE_CVDCERTSDIR`.
You may set this option with `cl_engine_set_str` and get it
with `cl_engine_get_str`, to override the compiled in default
CVD certs directory.
libfreshclam.so: Bump version to 4:0:0 (aka 4.0.0).
Add sigtool sign/verify tests and test certs.
Make it so downloadFile doesn't throw a warning if the server
doesn't have the .sign file.
Replace use of md5-based FP signatures in the unit tests with
sha256-based FP signatures because the md5 implementation used
by Python may be disabled in FIPS mode.
Fixes: https://github.com/Cisco-Talos/clamav/issues/1411
CMake: Add logic to enable the Rust openssl-sys / openssl-rs crates
to build against the same OpenSSL library as is used for the C build.
The Rust unit test application must also link directly with libcrypto
and libssl.
Fix some log messages with missing new lines.
Fix missing environment variable notes in --help messages and manpages.
Deconflict CONFDIR/DATADIR/CERTSDIR variable names that are defined in
clamav-config.h.in for libclamav from variable that had the same name
for use in clamav applications that use the optparser.
The 'clamav-test' certs for the unit tests will live for 10 years.
The 'clamav-beta.crt' public cert will only live for 120 days and will
be replaced before the stable release with a production 'clamav.crt'.
Store URLs found in HTML `<a>` and `<form>` tags during scan of HTML files
when recording scan metadata.
HTML URL recording will be ON by default, but is a part of the
generate-metadata-json feature.
The generate-metadata-json feature is OFF by default.
This introduces a new general scan option:
- libclamav: `CL_SCAN_GENERAL_STORE_HTML_URLS`.
- ClamD: `JsonStoreHTMLUrls`.
- ClamScan: `--json-store-html-urls`
Thank you Matt Jolly for the helpful comment on the pull request.
Add keys to the metadata.json file that informs the user that a scanned
ole2 file is encrypted. Information about the type of encryption is
provided when the information is available. This feature co-authored by
Micah Snyder.
The clamscan test "assorted_test.py::TC::test_pe_cert_trust" is about to
fail because the "test.exe" test file was signed with a cert set to
expire after only 2 years, and it has been 23 months.
While attempting to generate a new one that will last 73000 days (200
years), I discovered that any signing certificate set to expire after
2038 will fail the trust-check because the `ca.not_after` variable is
maxed out `time_t` incapable of expressing a higher number.
To fix this, I've upgraded the variables to `uint64_t`.
I also had to replace a bunch of generated signatures to match the new
"test.exe".
Finally, I noticed that "ca.not_before" was being set to the token[8]
instead of token[9], which presumably mean the "NotBefore" field for
Trusted and Revoked Certificates was non-functional, as it was treating
the "CertSign" boolean as the "NotBefore" value.
Fixes: https://github.com/Cisco-Talos/clamav/issues/1300
I found that the `url(data:` type does not matter to a browser.
In addition, whitespace may be placed in a few locations and the browser
will ignore it.
This commit accounts for this, and updates the test accordingly.
The PE cert test can be enabled now that the cert trust feature is
fixed. In so doing I found an issue with it -- it was also using the
block-certificate signature, which overrides the trust-certificate
signatures. This made me realize that we should also have a test to make
sure the block-cert signatures take predence over the trust-cert sigs.
I fixed the original sig and added this second test case.
Adds a test set authored by Andrew Williams that validates correct
allmatch behavior using as many features as possible to alert on a
test.exe program. Source for building the test.exe program is provided,
for those who are curious what it is and what it does, or in case it
needs to be re-built for some reason.
In addition to adding a test that verifies each of the sigs that should
alert, do alert, this adds a test to verify that if an authenticode
trust signature is added, none of the signatures alert. That test is
presently failing (expected failure, so the tests all pass) and should
be updated when the certificate verification bug is fixed.
Add tests:
- Test an import hash with wildcard size when all-match mode is
disabled.
- Test that bytecode rules will run after content match alerts in
all-match mode.
Add basic tests for the Container and Intermediates logical signature
features.
The Intermediates test verifies that a text file containing:
v1rusv1rus
and wrapped in a 7z and then zip archives can be correctly detected when
Intermediates is set to: CL_TYPE_7Z>CL_TYPE_ZIP
The Container test just checks the container is CL_TYPE_ZIP
If running multiple parallel processes of "xor_testfile.py" there was a
race condition between checking for the existence of the directory and
creating it. Now this is handled as a dependency in CMake.
Added a test to verify that clamscan can extract images from an XLS
document. The document has 2 images: a PNG and JPEG version of the
clamav demon/logo. The test requires the json metadata feature to verify
that the MD5 of the images are correct.
No other image formats were tested because despite the format allegedly
supporting other imate formats, Excel converts TIFF, BMP, and GIF images
to PNG files when you insert them.
The split test files are flagged by some AV's because they look like
broken executables. Instead of splitting the test files to prevent
detections, we should encrypt them. This commit replaces the "reassemble
testfiles" script with a basic "XOR testfiles" script that can be used
to encrypt or decrypt test files. This commit also of course then
replaces all the split files with xor'ed files.
The test and unit_tests directories were a bit of a mess, so I
reorganized them all into unit_tests with all of the test files placed
under "unit_tests/input" using subdirectories for different types of files.
290424 Missing break in switch - In hash_match: Missing break
statement between cases in switch statement
290414 Resource leak - In cli_scanishield_msi: Leak of memory or
pointers to system resources. Memory leak in a fail case
288197 Resource leak - In decrypt_any: Leak of memory or pointers
to system resources. Memory leak in a fail case
290426 Resource leak - In cli_magic_scan: Leak of memory or pointers
to system resources. Leaked a file prefix when running with
--save-temps
192923 Resource leak - In cli_scanrar: Leak of memory or pointers to
system resources. Leaked a file descriptor if a virus was found in
a RAR file comment
225146 Resource leak - In cli_scanegg: Leak of memory or pointers
to system resources. Leaked a file descriptor if unable to write
a comment file to disk
290425 Resource leak - In scan_common: Leak of memory or pointers
to system resources. Memory leaks in various fail cases.
Also changes cli_scanrar to write out the file comment only if
--leave-temps is specified and scan the buffer (like what is done
in cli_scanegg) instead of writing the file out, scanning that,
and then deleting the file if --leave-temps is not specified.
The unit tests stopped working when correcting an issue with a
switch statement that determined what type of signature had matched
on a Google SafeBrowsing GDB rule. Looking into the unit tests, it
looks like the code had always assumed that the test cases would be
detected by a malware test rule in unit_tests/input/daily.gdb, but
now some of the tests get matched on the phishing test rule.
I updated the test logic to be more clear, and added tests for both
cases now.
Fix some memory leaks in libclamav/scanners.c
