41 Commits

Author SHA1 Message Date
Jessica Stokes
d9fee3c460 WebClient: update to synthesise the user-agent using env variables 2024-07-20 14:42:13 -07:00
Jessica Stokes
2084648afe Add AGPL License
fixes #21
2024-07-20 14:42:13 -07:00
Jessica Stokes
87d397f2de Make sure permit world writable temp has correct copyright information 2024-07-20 14:42:13 -07:00
Jessica Stokes
7bff16a815 Expand error handling framework to quash expected errors 2023-01-02 11:27:45 -08:00
Jessica Stokes
c197daca3c Allow world-writable temp in cache clean script 2021-08-05 18:44:24 -07:00
Jessica Stokes
1b53b3e367 Make cache cleaning verbose 2021-08-04 13:17:44 -07:00
Jessica Stokes
9a6e6b4a06 Handle some particularly bizarre dates returned by the Wayback Machine 2021-08-01 21:47:13 -07:00
Jessica Stokes
bb63255d65 Add a script to allow cleaning the WebClient cache periodically 2021-08-01 20:48:32 -07:00
Jessica Stokes
92867ace6c Split out WebClient Cache module 2021-08-01 20:38:00 -07:00
Jessica Stokes
c76aa7f8dc Make pluralize function force numbers to be integers 2021-08-01 16:29:19 -07:00
Jessica Stokes
ed07256e71 Initial pass at handling wildcards and generating a site map 2021-08-01 12:42:22 -07:00
Jessica Stokes
48fef51428 Rubocop a bunch of stuff 2021-07-31 11:56:20 -07:00
Jessica Stokes
03ffb21ee3 Allow turning off the cache instead of using mocking 2021-07-31 11:45:15 -07:00
Jessica Stokes
d1ae5202ba Fix a warning from YAML.safe_load 2021-07-14 09:45:26 -07:00
Jessica Stokes
060df35d6d Make all CGI files self-calling modules so we can test them more easily 2021-06-01 20:41:09 -07:00
Jessica Stokes
482f604a45 Add a spec for the LegacyClientEncoding class 2021-05-29 15:37:57 -07:00
Jessica Stokes
ea9b8884f5 Add a spec for the CDX module, playing with spec'ing the whole thing 2021-05-29 13:02:38 -07:00
Jessica Stokes
b9791ae3e5 Add some utility functions for pluralisation and encoding overrides 2021-05-29 09:56:25 -07:00
Jessica Stokes
48e90c49c1 Add a TODO about curl's UTF-8 parameter behaviour 2021-05-29 09:45:47 -07:00
Jessica Stokes
a2829d04e8 Update outgoing HTTP request error handling 2021-05-28 21:15:31 -07:00
Jessica Stokes
7fcdf34899 Get Rubocopped 2021-05-28 19:34:58 -07:00
Jessica Stokes
2471131198 Move quotify into legacy encoding helper 2021-05-28 19:33:20 -07:00
Jessica Stokes
459bc199bd Move legacy encoding support into a class 2021-05-28 19:30:23 -07:00
Jessica Stokes
780ce02e67 Update error handling to hide unhandled errors, and show input errors 2021-05-28 19:19:31 -07:00
Jessica Stokes
d24b08f6d5 Implement a custom WebClient class, which caches web responses 2021-05-28 18:51:44 -07:00
Jessica Stokes
f6f9d13dd4 Ensure JSON is imported in the error reporting file 2021-05-28 18:50:36 -07:00
Jessica Stokes
1fc24a321e Move CDX functions to a class 2021-05-28 18:50:13 -07:00
Jessica Stokes
aedd201628 Gracefully handle zero-result pages 2021-05-28 17:17:07 -07:00
Jessica Stokes
466f864089 Run Rubocop's defaults over the whole thing 2021-05-24 17:25:26 -07:00
Jessica Stokes
1749427c15 Report errors to stderr instead 2021-05-24 16:49:16 -07:00
Jessica Stokes
6f27953090 Squelch Bugsnag's logging so CGI doesn't break 2021-05-24 15:07:50 -07:00
Jessica Stokes
7e64e42820 Integrate with Bugsnag so errors can be handled better 2021-05-24 14:12:39 -07:00
Jessica Stokes
f881856e58 Implement a patch to fix OpenURI on nearlyfreespeech.net hosting
NFSN's tmp directory is world-writable, but the CGI process runs as a user who is part of "world," which means Ruby's usual security protections here don't make sense. `FORCE_WORLD_WRITABLE_TEMP` should not be set in any normal circumstances, as it would cause a potentially significant security risk.
2021-05-24 13:00:18 -07:00
Jessica Stokes
f06792b7c2 Revert "Revert "Switch to OpenURI and provide a user-agent string for niceness""
This reverts commit 251b51753c.
2021-05-24 12:58:45 -07:00
Jessica Stokes
251b51753c Revert "Switch to OpenURI and provide a user-agent string for niceness"
This reverts commit b6908edfa2.
2021-05-24 12:14:02 -07:00
Jessica Stokes
b6908edfa2 Switch to OpenURI and provide a user-agent string for niceness 2021-05-23 23:41:27 -07:00
Jessica Stokes
86b24f48aa Add handling of user-agents which don't honour page encodings 2021-05-23 23:24:14 -07:00
Jessica Stokes
1a51f64131 Delete an unused utility function 2021-05-23 18:38:16 -07:00
Jessica Stokes
964297e01f Implement date selection in the history views 2021-05-22 21:23:08 -07:00
Jessica Stokes
330cd4084f Tabs to Spaces 2021-05-22 13:46:14 -07:00
Jessica Stokes
c694df26d1 Remove initial Rails implementation 2021-05-22 11:09:12 -07:00