- Added `label` computed attribute to `PhotoResource` to indicate the side of the seed packet.
- Introduced `needs_ocr` filter in `PhotoResource` to identify photos awaiting OCR processing.
- Updated `SeedResource` to associate photos with seeds.
- Modified `photos_request_spec` to include seed relationships and test for photo labels.
- Added filtering tests for photos needing OCR in `photos_request_spec`.
- Removed obsolete Rake task for listing photos needing OCR.
- Updated `read_packets.py` to fetch photos needing OCR from the API.
- Introduced a Rake task `packets:needing_ocr` to list photos of seed packets that lack text, outputting results as JSON lines.
- Created a Python script `read_packets.py` to process the output of the Rake task, downloading images and performing OCR to extract text.
- Updated model specs for `Crop` to ensure proper handling of photos associated with different seeds.
- Refactored API request specs for `Photos` and `Seeds`, removing unnecessary attributes from the response.
- Added tests for the new Rake task to ensure correct functionality and output format.
* Run photos:prune_404s task monthly on the 1st of the month
Assuming Heroku Scheduler invokes the task daily, check if today is
the first day of the month before pruning 404 photos. Also add spec to test this behavior.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Delete spec/tasks/photos_prune_404s_spec.rb
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
- Renamed all instances of 'Activity' to 'Job' in views, controllers, and models.
- Updated routes to reflect the change from activities to jobs.
- Modified database schema to rename the activities table to jobs and adjusted related columns.
- Updated locale files to change activity-related translations to jobs.
- Adjusted tests to ensure they reference jobs instead of activities.
- Removed the category field from jobs as they are now a plain list.
- Updated seed data and sample data generation to create jobs instead of activities.
- Deleted Elasticsearch gems from Gemfile and Gemfile.lock.
- Updated README and TECH documentation to reflect the removal of Elasticsearch.
- Removed Elasticsearch-related code from models, controllers, services, and factories.
- Adjusted tests to eliminate dependencies on Elasticsearch.
- Cleaned up Docker configuration and environment variables related to Elasticsearch.
- Added a TODO file for future gem reviews and potential removals.
* Remove unused bluecloth gem
Last released in 2011. It was only required, never called; kramdown
renders the markdown.
* Add specs for Tilt markdown rendering and posts markdown partial
- Updated MemberLocationsController to use a hash for location updates, ensuring coordinates are not geocoded again.
- Changed response status codes in member_locations_spec to use symbols for better readability.
- Modified PlacesController to use parentheses for defining MEMBER_FIELDS.
- Enhanced SuggestionsController to disable Rails/SkipsModelValidations for update_columns.
- Updated SeasonsHelper to use compact_blank for cleaner output.
- Refactored MemberDashboard service for improved readability and performance.
- Adjusted migration files to use parentheses for index definitions.
- Updated schema.rb to maintain consistency in index definitions.
- Added SampleData class to generate sample data for development.
- Cleaned up sample_data.rake task to utilize the new SampleData class.
- Updated tests in member_dashboard_spec and place_suggester_spec for consistency and readability.
* Optimize markdownify allocations with in-place gsub and combined regex
Avoid chaining multiple full-string copies during markdown parsing. Duplicate
the input string once in ApplicationHelper#markdownify and modify it in-place
using gsub! in GrowstuffMarkdown#expand_crops! and #expand_members!.
Combine MEMBER_REGEX and MEMBER_AT_REGEX with Regexp.union to parse member
mentions in a single pass.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Apply suggestion from @CloCkWeRX
* Apply suggestion from @CloCkWeRX
* Optimize markdownify allocations with in-place gsub and combined regex
Avoid chaining multiple full-string copies during markdown parsing. Duplicate
the input string once in ApplicationHelper#markdownify and modify it in-place
using gsub! in GrowstuffMarkdown#expand_crops! and #expand_members!.
Combine MEMBER_REGEX and MEMBER_AT_REGEX with Regexp.union to parse member
mentions in a single pass.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Add rake tasks to pull crops and a member's data from production
crops:pull_from_production copies approved crops, and
members:pull_from_production copies one member's gardens, plantings and
planting photos, from growstuff.org into a local database. Both refuse to
run in a production environment.
They are built to be gentle on a server that has been struggling: one
request at a time with a pause between requests, exponential backoff
(honouring Retry-After) on timeouts, 429s and 5xx, and a hard stop rather
than endless retries. The client also stops if a server keeps sending the
same page, and will not follow a next link to another host.
Gardens and plantings come from the site's own JSON, which carries every
column; the JSON:API only exposes a garden's name, and its related
resource endpoints return the wrong records. Only the crops that a
member's plantings use are fetched. Flickr photos are stored as records
pointing at Flickr's image URLs, as on production, so nothing is
downloaded from Flickr.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Add ACTIVE_ONLY option to the member import
ACTIVE_ONLY=true leaves out the site's all=1 flag, so production lists only
the member's active gardens and current (not finished, not failed)
plantings. The server does the filtering, so it also means fewer requests,
and no crops or photos are fetched for plantings that are skipped.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Refactor Plantings to remove Elasticsearch dependency
This commit refactors the Plantings controller, model, and views to use
ActiveRecord instead of Searchkick/Elasticsearch.
Key changes:
- Refactored `PlantingsController#index` to use an ActiveRecord-based
`plantings` method.
- Moved `homepage_records` logic from `SearchPlantings` concern to the
`Planting` model using ActiveRecord scopes.
- Removed `SearchPlantings` concern and deleted the file.
- Updated `home/_plantings.html.haml` and `plantings/index.rss.haml` to
use dot notation for `Planting` object attributes.
- Updated specs to remove Searchkick-related setup and assertions.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix Planting.reindex
* Fix test
* Mark test pending
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Refactor Seeds to remove Elasticsearch dependency
This commit refactors the `SeedsController` and `Seed` model to remove
dependency on Elasticsearch and Searchkick.
Key changes:
- Removed `SearchSeeds` concern from the `Seed` model.
- Reimplemented `Seed.homepage_records` using ActiveRecord.
- Refactored `SeedsController#index` to use ActiveRecord for filtering,
eager loading, and pagination.
- Deleted `app/models/concerns/search_seeds.rb`.
- Updated relevant specs to remove Searchkick-specific tags and setup.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Remove Seed.reindex
* Remove trait
* Fix tests
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
- Added `send_harvest_reminder` preference to Member model and settings UI.
- Implemented `harvest_in_next_week?` in PredictHarvest concern.
- Created `harvest_reminder` email with localized templates.
- Added `growstuff:send_harvest_reminders` Rake task to run weekly.
- Refactored existing and new reminder tasks to use `deliver_later` for scalability.
- Added unit tests for prediction logic and mailer.
- Fixed a bug in the existing planting reminder task where it was using an uninitialized constant `Notifier`.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Refactor Activity model to remove Elasticsearch integration
- Removed `SearchActivities` concern and Searchkick from `Activity` model.
- Implemented `Activity.homepage_records` using ActiveRecord with `DISTINCT ON` for PostgreSQL.
- Updated `ActivitiesController#index` to use ActiveRecord queries with eager loading and pagination.
- Added `active` scope to `Activity`.
- Added unit tests for `Activity` model.
- Deleted `app/models/concerns/search_activities.rb`.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix NoMethodError: undefined method 'reindex' for class Activity
- Removed all calls to `Activity.reindex` in migrations, rake tasks, and spec helpers.
- These were causing failures after the removal of Searchkick from the Activity model.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix ambiguous column id in homepage_records query
- Updated `Activity.homepage_records` to use `activities.id` instead of `id` in the subquery.
- This resolves the `PG::AmbiguousColumn: ERROR: column reference "id" is ambiguous` error.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix ambiguous created_at in homepage_records query
- Use `unscoped` in the subquery for `Activity.homepage_records` to bypass the default scope from `Ownable` concern.
- This prevents the join with the `members` table in the subquery, which was causing `PG::AmbiguousColumn: ERROR: column reference "created_at" is ambiguous`.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Complete refactoring of Activity model to remove Elasticsearch
- Removed SearchActivities concern and searchkick integration.
- Updated ActivitiesController#index to use ActiveRecord queries.
- Implemented performant Activity.homepage_records using DISTINCT ON (PostgreSQL).
- Added Activity.active scope.
- Added no-op Activity.reindex (class and instance methods) for backward compatibility.
- Cleaned up leftover reindex calls in rake tasks, migrations, and spec helpers.
- Added unit tests for new Activity model logic.
- Updated factories to include no-op reindex traits.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Less eager loading
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Add members:cleanup_inactive rake task
This task identifies and deletes members who have not logged in for over
24 months and have no gardens, plantings, or other activity (posts,
comments, seeds, harvests, etc).
Includes support for DRY_RUN=true to preview deletions.
Added tests in spec/tasks/members_spec.rb.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Refactor activity check to Member#has_activity? and update rake task
- Added `Member#has_activity?` to encapsulate the check for gardens, plantings, and other activity.
- Updated `members:cleanup_inactive` rake task to use `Member#has_activity?`.
- Maintained `DRY_RUN` support and existing tests.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Apply suggestion from @CloCkWeRX
* Apply suggestions from code review
Co-authored-by: Daniel O'Connor <daniel.oconnor@gmail.com>
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Improves the `wikidata:import_alternate_names` Rake task to first use the `wikidata_id` from the `scientific_names` table.
- Checks for a `wikidata_id` on a crop's `scientific_names`.
- Falls back to the existing `en_wikipedia_url` if no `wikidata_id` is found.
- Adds logging to indicate the source of the Wikidata ID.
- Improves error handling.
Improves the `wikidata:import_alternate_names` Rake task to first look up a crop's Wikidata ID using its scientific names.
- Iterates through a crop's scientific names to find a Wikidata ID.
- Falls back to the existing `en_wikipedia_url` if no ID is found via scientific names.
- Adds more detailed logging to the task.
- Improves error handling by checking for non-existent Wikipedia pages and using `.dig` for safer hash access.
* I will add the international alternate names for the crops.
* Mark required
* Update factory
* Add placeholder
* Fix seeds
* Add language, though hardcoded to EN in most places
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Daniel O'Connor <daniel.oconnor@gmail.com>
This change removes the twitter authentication feature from the application.
It removes the `omniauth-twitter` gem and all related code from controllers, views, and tests. It also removes the twitter icon and environment variable settings.
The `delete_pictures` rake task in `openfarm.rake` is amended to
also remove Photos where the `fullsize_url` starts with
`https://s3.amazonaws.com/openfarm-project/`.
This change helps in cleaning up legacy photos that are no longer
needed. The task description has also been updated to reflect this
change.
Brenda Wallace