* Optimize markdownify allocations with in-place gsub and combined regex
Avoid chaining multiple full-string copies during markdown parsing. Duplicate
the input string once in ApplicationHelper#markdownify and modify it in-place
using gsub! in GrowstuffMarkdown#expand_crops! and #expand_members!.
Combine MEMBER_REGEX and MEMBER_AT_REGEX with Regexp.union to parse member
mentions in a single pass.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Apply suggestion from @CloCkWeRX
* Apply suggestion from @CloCkWeRX
* Optimize markdownify allocations with in-place gsub and combined regex
Avoid chaining multiple full-string copies during markdown parsing. Duplicate
the input string once in ApplicationHelper#markdownify and modify it in-place
using gsub! in GrowstuffMarkdown#expand_crops! and #expand_members!.
Combine MEMBER_REGEX and MEMBER_AT_REGEX with Regexp.union to parse member
mentions in a single pass.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Add rake tasks to pull crops and a member's data from production
crops:pull_from_production copies approved crops, and
members:pull_from_production copies one member's gardens, plantings and
planting photos, from growstuff.org into a local database. Both refuse to
run in a production environment.
They are built to be gentle on a server that has been struggling: one
request at a time with a pause between requests, exponential backoff
(honouring Retry-After) on timeouts, 429s and 5xx, and a hard stop rather
than endless retries. The client also stops if a server keeps sending the
same page, and will not follow a next link to another host.
Gardens and plantings come from the site's own JSON, which carries every
column; the JSON:API only exposes a garden's name, and its related
resource endpoints return the wrong records. Only the crops that a
member's plantings use are fetched. Flickr photos are stored as records
pointing at Flickr's image URLs, as on production, so nothing is
downloaded from Flickr.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Add ACTIVE_ONLY option to the member import
ACTIVE_ONLY=true leaves out the site's all=1 flag, so production lists only
the member's active gardens and current (not finished, not failed)
plantings. The server does the filtering, so it also means fewer requests,
and no crops or photos are fetched for plantings that are skipped.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Refactor Plantings to remove Elasticsearch dependency
This commit refactors the Plantings controller, model, and views to use
ActiveRecord instead of Searchkick/Elasticsearch.
Key changes:
- Refactored `PlantingsController#index` to use an ActiveRecord-based
`plantings` method.
- Moved `homepage_records` logic from `SearchPlantings` concern to the
`Planting` model using ActiveRecord scopes.
- Removed `SearchPlantings` concern and deleted the file.
- Updated `home/_plantings.html.haml` and `plantings/index.rss.haml` to
use dot notation for `Planting` object attributes.
- Updated specs to remove Searchkick-related setup and assertions.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix Planting.reindex
* Fix test
* Mark test pending
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Refactor Seeds to remove Elasticsearch dependency
This commit refactors the `SeedsController` and `Seed` model to remove
dependency on Elasticsearch and Searchkick.
Key changes:
- Removed `SearchSeeds` concern from the `Seed` model.
- Reimplemented `Seed.homepage_records` using ActiveRecord.
- Refactored `SeedsController#index` to use ActiveRecord for filtering,
eager loading, and pagination.
- Deleted `app/models/concerns/search_seeds.rb`.
- Updated relevant specs to remove Searchkick-specific tags and setup.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Remove Seed.reindex
* Remove trait
* Fix tests
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
- Added `send_harvest_reminder` preference to Member model and settings UI.
- Implemented `harvest_in_next_week?` in PredictHarvest concern.
- Created `harvest_reminder` email with localized templates.
- Added `growstuff:send_harvest_reminders` Rake task to run weekly.
- Refactored existing and new reminder tasks to use `deliver_later` for scalability.
- Added unit tests for prediction logic and mailer.
- Fixed a bug in the existing planting reminder task where it was using an uninitialized constant `Notifier`.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Refactor Activity model to remove Elasticsearch integration
- Removed `SearchActivities` concern and Searchkick from `Activity` model.
- Implemented `Activity.homepage_records` using ActiveRecord with `DISTINCT ON` for PostgreSQL.
- Updated `ActivitiesController#index` to use ActiveRecord queries with eager loading and pagination.
- Added `active` scope to `Activity`.
- Added unit tests for `Activity` model.
- Deleted `app/models/concerns/search_activities.rb`.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix NoMethodError: undefined method 'reindex' for class Activity
- Removed all calls to `Activity.reindex` in migrations, rake tasks, and spec helpers.
- These were causing failures after the removal of Searchkick from the Activity model.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix ambiguous column id in homepage_records query
- Updated `Activity.homepage_records` to use `activities.id` instead of `id` in the subquery.
- This resolves the `PG::AmbiguousColumn: ERROR: column reference "id" is ambiguous` error.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Fix ambiguous created_at in homepage_records query
- Use `unscoped` in the subquery for `Activity.homepage_records` to bypass the default scope from `Ownable` concern.
- This prevents the join with the `members` table in the subquery, which was causing `PG::AmbiguousColumn: ERROR: column reference "created_at" is ambiguous`.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Complete refactoring of Activity model to remove Elasticsearch
- Removed SearchActivities concern and searchkick integration.
- Updated ActivitiesController#index to use ActiveRecord queries.
- Implemented performant Activity.homepage_records using DISTINCT ON (PostgreSQL).
- Added Activity.active scope.
- Added no-op Activity.reindex (class and instance methods) for backward compatibility.
- Cleaned up leftover reindex calls in rake tasks, migrations, and spec helpers.
- Added unit tests for new Activity model logic.
- Updated factories to include no-op reindex traits.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Less eager loading
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
* Add members:cleanup_inactive rake task
This task identifies and deletes members who have not logged in for over
24 months and have no gardens, plantings, or other activity (posts,
comments, seeds, harvests, etc).
Includes support for DRY_RUN=true to preview deletions.
Added tests in spec/tasks/members_spec.rb.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Refactor activity check to Member#has_activity? and update rake task
- Added `Member#has_activity?` to encapsulate the check for gardens, plantings, and other activity.
- Updated `members:cleanup_inactive` rake task to use `Member#has_activity?`.
- Maintained `DRY_RUN` support and existing tests.
Co-authored-by: CloCkWeRX <365751+CloCkWeRX@users.noreply.github.com>
* Apply suggestion from @CloCkWeRX
* Apply suggestions from code review
Co-authored-by: Daniel O'Connor <daniel.oconnor@gmail.com>
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Improves the `wikidata:import_alternate_names` Rake task to first use the `wikidata_id` from the `scientific_names` table.
- Checks for a `wikidata_id` on a crop's `scientific_names`.
- Falls back to the existing `en_wikipedia_url` if no `wikidata_id` is found.
- Adds logging to indicate the source of the Wikidata ID.
- Improves error handling.
Improves the `wikidata:import_alternate_names` Rake task to first look up a crop's Wikidata ID using its scientific names.
- Iterates through a crop's scientific names to find a Wikidata ID.
- Falls back to the existing `en_wikipedia_url` if no ID is found via scientific names.
- Adds more detailed logging to the task.
- Improves error handling by checking for non-existent Wikipedia pages and using `.dig` for safer hash access.
* I will add the international alternate names for the crops.
* Mark required
* Update factory
* Add placeholder
* Fix seeds
* Add language, though hardcoded to EN in most places
---------
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Daniel O'Connor <daniel.oconnor@gmail.com>
This change removes the twitter authentication feature from the application.
It removes the `omniauth-twitter` gem and all related code from controllers, views, and tests. It also removes the twitter icon and environment variable settings.
The `delete_pictures` rake task in `openfarm.rake` is amended to
also remove Photos where the `fullsize_url` starts with
`https://s3.amazonaws.com/openfarm-project/`.
This change helps in cleaning up legacy photos that are no longer
needed. The task description has also been updated to reflect this
change.
