♻️ Put token settings under config flag (#6551)

* ✨ Visual indicators subscription for teams and project settings

* 📎 Fixes PR feedback

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>

.circleci/config.yml

@@ -263,6 +263,12 @@ jobs:
           command: |
             cargo fmt --check
 
+      - run:
+          name: "lint"
+          working_directory: "./render-wasm"
+          command: |
+            ./lint
+
       - run:
           name: "cargo tests"
           working_directory: "./render-wasm"

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,29 +1,19 @@
-<!--
+### Related Ticket
 
-Some key notes before you open a PR:
+<!-- Reference the related GitHub/Taiga ticket. -->
 
- 1. Select which branch should this PR be merged in? By default, you should always merge to the develop branch.
- 2. PR name follows [convention](http://karma-runner.github.io/4.0/dev/git-commit-msg.html)
- 3. All tests pass locally, UI and Unit tests
- 4. All business logic and validations must be on the server-side
- 5. Update necessary Documentation
- 6. Put `closes #XXXX` in your comment to auto-close the issue that your PR fixes
+### Summary
 
+### Steps to reproduce 
 
-Also, if you're new here
+### Checklist
 
-- Contribution Guide => https://github.com/uxbox/uxbox/blob/develop/CONTRIBUTING.md
+- [ ] Choose the correct target branch; use `develop` by default.
+- [ ] Provide a brief summary of the changes introduced.
+- [ ] Add a detailed explanation of how to reproduce the issue and/or verify the fix, if applicable.
+- [ ] Include screenshots or videos, if applicable.
+- [ ] Add or modify existing integration tests in case of bugs or new features, if applicable.
+- [ ] Check CI passes successfully.
+- [ ] Update the `CHANGES.md` file, referencing the related GitHub issue, if applicable.
 
--->
-
-> Please provide enough information so that others can review your pull request:
-
-<!-- You can skip this if you're fixing a typo or updating existing documentation -->
-
-> Explain the **details** for making this change. What existing problem does the pull request solve?
-
-<!-- Example: When "Adding a function to do X", explain why it is necessary to have a way to do X. -->
-
-> Screenshots/GIFs
-
-<!-- Add images/recordings to better visualize the change: expected/current behviour -->
+<!-- For more details, check the contribution guidelines: https://github.com/penpot/penpot/blob/develop/CONTRIBUTING.md -->

.github/workflows/commit-checker.yml

@@ -26,7 +26,7 @@ jobs:
       - name: Check Commit Type
         uses: gsactions/commit-message-checker@v2
         with:
-          pattern: '^:(lipstick|globe_with_meridians|wrench|books|arrow_up|arrow_down|zap|ambulance|construction|boom|fire|whale|bug|sparkles|paperclip|tada|recycle):\s[A-Z].*[^.]$'
+          pattern: '^(Merge|:(lipstick|globe_with_meridians|wrench|books|arrow_up|arrow_down|zap|ambulance|construction|boom|fire|whale|bug|sparkles|paperclip|tada|recycle):)\s[A-Z].*[^.]$'
           flags: 'gm'
           error: 'Commit should match CONTRIBUTING.md guideline'
           checkAllCommitMessages: 'true' # optional: this checks all commits associated with a pull request

.gitignore

@@ -68,6 +68,8 @@
 /vendor/**/target
 /vendor/svgclean/bundle*.js
 /web
+/library/target/
+
 clj-profiler/
 node_modules
 /test-results/

CHANGES.md

@@ -1,6 +1,93 @@
 # CHANGELOG
 
-## 2.5.2
+## 2.8.0 (Next / Unreleased)
+
+### :rocket: Epics and highlights
+
+### :boom: Breaking changes & Deprecations
+
+**Breaking changes on penpot library:**
+
+The library entrypoint API object has been changed. From now you start creating a new
+build context, from where you can add multiple files and attach media. This change add the
+ability to build more than one file at same time and export them in an unique .penpot
+file.
+
+```js
+const context = penpot.createBuildContext()
+
+context.addFile({name:"aa"})
+context.addPage({name:"aa"})
+context.closePage()
+context.closeFile()
+
+;; barray is instance of Uint8Array
+const barray = penpot.exportAsBytes(context);
+```
+
+The previous `file.export()` method has been removed and several alternatives are
+added as first level functions on penpot library API entrypoint:
+
+- `exportAsBytes(BuildContext context) -> Promise<Uint8Array>`
+- `exportAsBlob(BuildContext context) -> Promise<Blob>`
+- `exportStream(BuildContext context, WritableStream stream) -> Promise<Void>`
+
+The stream variant allows writting data as it is generated to the stream, without the need
+to store the generated output entirelly in the memory.
+
+There are also relevant semantic changes in how components should be created: this
+refactor removes all notions of the old components (v1). Since v2, the shapes that are
+part of a component live on a page. So, from now on, to create a component, you should
+first create a frame, then add shapes and/or groups to that frame, and then create a
+component by declaring that frame as the component root.
+
+A non exhaustive list of changes:
+
+- Change the signature of the `addPage` method: it now accepts an object (as a single argument) where you can pass `id`,
+  `name`, and `background` props (instead of the previous positional arguments)
+- Rename the `createRect` method to `addRect`
+- Rename the `createCircle` method to `addCircle`
+- Rename the `createPath` method to `addPath`
+- Rename the `createText` method to `addText`
+- Rename the `addArtboard` method to `addBoard`
+- Rename `startComponent` to `addComponent` (to preserve the naming style)
+- Rename `createComponentInstance` to `addComponentInstance` (to preserve the naming style)
+- Remove `lookupShape`
+- Remove `asMap`
+- Remove `updateLibraryColor` (use `addLibraryColor` if you just need to replace a color)
+- Remove `deleteLibraryColor` (this library is intended to build files)
+- Remove `updateLibraryTypography` (use `addLibraryTypography` if you just need to replace a typography)
+- Remove `deleteLibraryTypography` (this library is intended to build files)
+- Remove `add/update/deleteLibraryMedia` (they are no longer supported by Penpot and have been replaced by components)
+- Remove `deleteObject` (this library is intended to build files)
+- Remove `updateObject` (this library is intended to build files)
+- Remove `finishComponent` (it is no longer necessary; see below for more details on component creation changes)
+
+- Change the `getCurrentPageId` function to a read-only `currentPageId` property
+- Add `currentFileId` read-only property
+- Add `currentFrameId` read-only property
+- Add `lastId` read-only property
+
+### :heart: Community contributions (Thank you!)
+
+### :sparkles: New features
+
+- Optimize profile setup flow for better user experience [Taiga #10028](https://tree.taiga.io/project/penpot/us/10028)
+- Update base image for Docker Backend and Exporter to Ubuntu 24.04
+- Update base image for Docker Frontend to Nginx 1.28.0
+- Allow multi file token import [Github #27](https://github.com/tokens-studio/penpot/issues/27)
+- Create `input*` wrapper component, and `label*`, `input-field*` and `hint-message*` components [Taiga #10713](https://tree.taiga.io/project/penpot/us/10713)
+- Deselect layers (and path nodes) with Ctrl+Shift+Drag [Github #2509](https://github.com/penpot/penpot/issues/2509)
+- Copy to SVG from contextual menu [Github #838](https://github.com/penpot/penpot/issues/838)
+- Add styles for Inkeep Chat at workspace [Taiga #10708](https://tree.taiga.io/project/penpot/us/10708)
+
+### :bug: Bugs fixed
+
+- Fix spacing / sizes of different elements in the measurements section of the design tab [Taiga #11076](https://tree.taiga.io/project/penpot/issue/11076)
+- Fix selection of short paths [Github #4472](https://github.com/penpot/penpot/issues/4472)
+- Fix element positioning on the right side to adjust to grid [#11073](https://tree.taiga.io/project/penpot/issue/11073)
+
+## 2.7.0 (Unreleased)
 
 ### :rocket: Epics and highlights
 
@@ -8,7 +95,155 @@
 
 ### :heart: Community contributions (Thank you!)
 
+- Design improvements to the Invitations page with an empty state [GitHub #2608](https://github.com/penpot/penpot/issues/2608) by [@iprithvitharun](https://github.com/iprithvitharun)
+
 ### :sparkles: New features
+
+- Update board presets with a newer devices [Taiga #10610](https://tree.taiga.io/project/penpot/us/10610)
+- Propagate "sharing a prototype" to editors and viewers [Taiga #8853](https://tree.taiga.io/project/penpot/us/8853)
+- Design improvements to the Invitations page with an empty state [Taiga #4554](https://tree.taiga.io/project/penpot/us/4554)
+- Duplicate token sets [Taiga #10694](https://tree.taiga.io/project/penpot/issue/10694)
+- Add set selection in create Token themes flow [Taiga #10746](https://tree.taiga.io/project/penpot/issue/10746)
+- Display indicator on not active sets [Taiga #10668](https://tree.taiga.io/project/penpot/issue/10668)
+
+### :bug: Bugs fixed
+
+- Fix "at" icon to match all icons on app [Taiga #11136](https://tree.taiga.io/project/penpot/issue/11136)
+- Fix problem in viewer with the back button [Taiga #10907](https://tree.taiga.io/project/penpot/issue/10907)
+- Fix resize bar background on tokens panel [Taiga #10811](https://tree.taiga.io/project/penpot/issue/10811)
+- Fix shortcut for history version panel [Taiga #11006](https://tree.taiga.io/project/penpot/issue/11006)
+- Fix positioning of comment drafts when near the right / bottom edges of viewport [Taiga #10534](https://tree.taiga.io/project/penpot/issue/10534)
+- Fix path having a wrong selrect [Taiga #10257](https://tree.taiga.io/project/penpot/issue/10257)
+- Fix SVG `stroke-linecap` property when importing SVGs [Taiga #9489](https://tree.taiga.io/project/penpot/issue/9489)
+- Fix position problems cutting-pasting a component [Taiga #10677](https://tree.taiga.io/project/penpot/issue/10677)
+- Fix design tab has a horizontal scroll [Taiga #10660](https://tree.taiga.io/project/penpot/issue/10660)
+- Fix long file names being clipped when longer than allowed length [Taiga #10662](https://tree.taiga.io/project/penpot/issue/10662)
+- Fix problem with error detail in toast [Taiga #10519](https://tree.taiga.io/project/penpot/issue/10519)
+- Fix view mode error when an external user tries to export something from a prototype using a shared link [Taiga #10251](https://tree.taiga.io/project/penpot/issue/10251)
+- Fix merge path nodes with only one node selected [Taiga #9626](https://tree.taiga.io/project/penpot/issue/9626)
+- Fix problem with import errors [Taiga #10040](https://tree.taiga.io/project/penpot/issue/10040)
+- Fix color gradient on texts [Taiga Issue #7488](https://tree.taiga.io/project/penpot/issue/7488)
+- Add support for self mentions [Taiga #10809](https://tree.taiga.io/project/penpot/issue/10809)
+- Fix team info settings alignment [Taiga #10869](https://tree.taiga.io/project/penpot/issue/10869)
+- Fix left sidebar horizontal scroll on nested layers [Taiga #10791](https://tree.taiga.io/project/penpot/issue/10791)
+- Improve error message details importing tokens [Taiga Issue #10772](https://tree.taiga.io/project/penpot/issue/10772)
+- Fix no selected set after Drag & Drop [Github #71](https://github.com/tokens-studio/penpot/issues/71)
+- Styledictionary v5 Update [Github #6283](https://github.com/penpot/penpot/pull/6283)
+- Fix Rename a set throws an internal error [Github #78](https://github.com/tokens-studio/penpot/issues/78)
+- Fix Out of Sync Token Value & Color Picker [Github #102](https://github.com/tokens-studio/penpot/issues/102)
+- Fix Color should preserve color space [Github #69](https://github.com/tokens-studio/penpot/issues/69)
+- Fix cannot rename Design Token Sets when group of same name exists [Taiga Issue #10773](https://tree.taiga.io/project/penpot/issue/10773)
+- Fix problem when duplicating grid layout [Github #6391](https://github.com/penpot/penpot/issues/6391)
+- Fix issue that makes workspace shortcuts stop working [Taiga #11062](https://tree.taiga.io/project/penpot/issue/11062)
+- Fix problem while syncing library colors and typographies [Taiga #11068](https://tree.taiga.io/project/penpot/issue/11068)
+- Fix problem with path edition of shapes [Taiga #9496](https://tree.taiga.io/project/penpot/issue/9496)
+- Fix exception on paste invalid html [Taiga #11047](https://tree.taiga.io/project/penpot/issue/11047)
+- Fix share button being displayed with no permissions [Taiga #11086](https://tree.taiga.io/project/penpot/issue/11086)
+- Fix inline styles in code tab [Taiga Issue #7583](https://tree.taiga.io/project/penpot/issue/7583)
+- Fix exception on returning openapi.json
+- Fix json encoding of TokensLib [Taiga #10994](https://tree.taiga.io/project/penpot/issue/10994)
+
+## 2.6.2
+
+### :bug: Bugs fixed
+
+- Increase the height of the right sidebar dropdowns [Taiga #10615](https://tree.taiga.io/project/penpot/issue/10615)
+- Fix scroll on token themes modal [Taiga #10745](https://tree.taiga.io/project/penpot/issue/10745)
+- Fix collapsing grouped sets in "edit Theme" closes the dialog [Taiga #10771](https://tree.taiga.io/project/penpot/issue/10771)
+- Fix unexpected exception on path editor on merge segments when undo stack is empty
+- Fix pricing CTA to be under a config flag [Taiga #10808](https://tree.taiga.io/project/penpot/issue/10808)
+- Fix allow moving a main component into another [Taiga #10818](https://tree.taiga.io/project/penpot/issue/10818)
+- Fix several issues with internal srepl helpers
+- Fix unexpected exception on template import from libraries
+- Fix incorrect uuid parsing from different parts of code
+- Fix update layout on component restore [Taiga #10637](https://tree.taiga.io/project/penpot/issue/10637)
+- Fix horizontal scroll in viewer [Github #6290](https://github.com/penpot/penpot/issues/6290)
+- Fix detach component in a particular case [Taiga #10837](https://tree.taiga.io/project/penpot/issue/10837)
+
+## 2.6.1
+
+### :bug: Bugs fixed
+
+- Fix webhooks not shown in list [Taiga #10763](https://tree.taiga.io/project/penpot/issue/10763)
+- Fix colorpicker scroll when dropdown displayed [Taiga #10696](https://tree.taiga.io/project/penpot/issue/10696)
+- Clean internal workspace state on exit or url changed [Taiga #10619](https://tree.taiga.io/project/penpot/issue/10619)
+
+## 2.6.0
+
+### :rocket: Epics and highlights
+
+- Design Tokens
+
+### :boom: Breaking changes & Deprecations
+
+### :heart: Community contributions (Thank you!)
+
+### :sparkles: New features
+
+- [COMMENTS] "Mark All as Read" Functionality in Dashboard [Taiga #9235](https://tree.taiga.io/project/penpot/us/9235)
+- [COMMENTS] Bubble Groups [Taiga #9236](https://tree.taiga.io/project/penpot/us/9236)
+- Change templates carrousel [Taiga #9803](https://tree.taiga.io/project/penpot/us/9803)
+- [DESIGN TOKENS] Tokens CRUD. Types added: Color, Opacity, Border radius, Dimension, Sizing, Spacing, Rotation and Stroke.
+- [DESIGN TOKENS] Create references (alias) that point to other tokens.
+- [DESIGN TOKENS] Math operations in token values.
+- [DESIGN TOKENS] Sets CRUD, grouping and reordering.
+- [DESIGN TOKENS] Multidimensional Themes and Sets management.
+- [DESIGN TOKENS] Apply/Remove tokens to/from elements from the Tokens tab.
+- [DESIGN TOKENS] Integration with components.
+- [DESIGN TOKENS] Import and export tokens from a JSON file.
+- [DESIGN TOKENS] Apply Themes and Sets at document level.
+- Add more descriptive tooltip to boards for first time users [Taiga #9426](https://tree.taiga.io/project/penpot/us/9426)
+- First State of a Project Changes Consolidation [Taia #10605](https://tree.taiga.io/project/penpot/us/10605)
+
+### :bug: Bugs fixed
+
+- Fix opacity in frame containers [Github #5858](https://github.com/penpot/penpot/pull/5858)
+- Avoid resizing on click [Taiga #10213](https://tree.taiga.io/project/penpot/issue/10213)
+- Hide horizontal scroll from dashboard sidebar [Taiga #10422](https://tree.taiga.io/project/penpot/issue/10422)
+- Fix cut and paste a copy a cmponent inside its parent [Taiga #10365](https://tree.taiga.io/project/penpot/us/10365)
+- Fix duplicate page with component over frame [Taiga #8151](https://tree.taiga.io/project/penpot/issue/8151) and [Taiga #9698](https://tree.taiga.io/project/penpot/issue/9698)
+- The plugin list in the navigation menu lacks scrolling, some plugins are not visible when a large number are installed [Taiga #9360](https://tree.taiga.io/project/penpot/us/9360)
+- Fix hidden toolbar click event still available [Taiga #10437](https://tree.taiga.io/project/penpot/us/10437)
+- Fix hovering over templates [Taiga #10545](https://tree.taiga.io/project/penpot/issue/10545)
+- Fix problem with default shadows value in plugins [Plugins #191](https://github.com/penpot/penpot-plugins/issues/191)
+- Fix problem with constraints when creating group [Taiga #10455](https://tree.taiga.io/project/penpot/issue/10455)
+- Fix opening pen with shortcut multiple times breaks toolbar [Taiga #10566](https://tree.taiga.io/project/penpot/issue/10566)
+- Fix actions when workspace is visited first time [Taiga #10548](https://tree.taiga.io/project/penpot/issue/10548)
+- Chat icon overlaps "Show" button in carrousel section [Taiga #10542](https://tree.taiga.io/project/penpot/issue/10542)
+- Fix assets name on inspect tab [Taiga #10630](https://tree.taiga.io/project/penpot/issue/10630)
+- Fix chat icon overlaps "Show" button in carrousel section [Taiga #10542](https://tree.taiga.io/project/penpot/issue/10542)
+- Fix incorrect handling of background task result (now task rows are properly marked as completed)
+- Fix available size of resize handler [Taiga #10639](https://tree.taiga.io/project/penpot/issue/10639)
+- Internal error when install a plugin by penpothub - Try plugin [Taiga #10542](https://tree.taiga.io/project/penpot/issue/10542)
+- Add character limitation to asset inputs [Taiga #10669](https://tree.taiga.io/project/penpot/issue/10669)
+- Fix Storybook link 'list of all available icons' wrong path [Taiga #10705](https://tree.taiga.io/project/penpot/issue/10705)
+
+
+## 2.5.4
+
+### :heart: Community contributions (Thank you!)
+
+- Add support for WEBP format on shape export [Github #6053](https://github.com/penpot/penpot/pull/6053) and [Github #6074](https://github.com/penpot/penpot/pull/6074)
+
+### :bug: Bugs fixed
+
+- Fix feature loading on workspace when opening a file in a background
+  tab [Taiga #10377](https://tree.taiga.io/project/penpot/issue/10377)
+- Fix minor inconsistencies on RPC `get-file-libraries` and `get-file`
+  methods (add missing team-id prop)
+- Fix problem with viewer role and inspect mode [Taiga #9751](https://tree.taiga.io/project/penpot/issue/9751)
+- Fix error when clicking on a comment at the viewer's sidebar [Taiga #10465](https://tree.taiga.io/project/penpot/issue/10465)
+
+## 2.5.3
+
+### :bug: Bugs fixed
+
+- Component sync issues with multiple tabs [Taiga #10471](https://tree.taiga.io/project/penpot/issue/10471)
+
+## 2.5.2
+
+### :sparkles: New features
+
 - When the workspace is empty, set default the board creation tool [Taiga #9425](https://tree.taiga.io/project/penpot/us/9425)
 
 ### :bug: Bugs fixed
@@ -20,22 +255,12 @@
 
 ## 2.5.1
 
-### :rocket: Epics and highlights
-
-### :boom: Breaking changes & Deprecations
-
-### :heart: Community contributions (Thank you!)
-
 ### :sparkles: New features
 
 - Improve Nginx entryponit to get the resolvers dinamically by default
 
-### :bug: Bugs fixed
-
 ## 2.5.0
 
-### :rocket: Epics and highlights
-
 ### :boom: Breaking changes & Deprecations
 
 Although this is not a breaking change, we believe it’s important to highlight it in this
@@ -64,9 +289,6 @@ If you have a big database and many cores available, you can reduce the time of
 all files by increasing paralelizacion changing the `max-jobs` value from 1 to N (where N
 is a number of cores)
 
-
-### :heart: Community contributions (Thank you!)
-
 ### :sparkles: New features
 
 - [GRADIENTS] New gradients UI with multi-stop support. [Taiga #3418](https://tree.taiga.io/project/penpot/epic/3418)
@@ -83,6 +305,7 @@ is a number of cores)
 
 ### :bug: Bugs fixed
 
+- Fix menu shadow color [Taiga #10102](https://tree.taiga.io/project/penpot/issue/10102)
 - Fix missing state refresh on notifications update [Taiga #10253](https://tree.taiga.io/project/penpot/issue/10253)
 - Fix icon visualization on select component [Taiga #8889](https://tree.taiga.io/project/penpot/issue/8889)
 - Fix typo on integration tests docs [Taiga #10112](https://tree.taiga.io/project/penpot/issue/10112)

CONTRIBUTING.md

@@ -1,62 +1,59 @@
 # Contributing Guide #
 
 Thank you for your interest in contributing to Penpot. This is a
-generic guide that details how to contribute to Penpot in a way that
-is efficient for everyone. If you want a specific documentation for
-different parts of the platform, please refer to `docs/` directory.
-
+generic guide that details how to contribute to the project in a way that
+is efficient for everyone. If you are looking for specific documentation on
+different parts of the platform, please refer to the `docs/` directory,
+or the rendered version at the [Help Center](https://help.penpot.app/).
 
 ## Reporting Bugs ##
 
 We are using [GitHub Issues](https://github.com/penpot/penpot/issues)
-for our public bugs. We keep a close eye on this and try to make it
+for our public bugs. We keep a close eye on them and try to make it
 clear when we have an internal fix in progress. Before filing a new
 task, try to make sure your problem doesn't already exist.
 
-If you found a bug, please report it, as far as possible with:
+If you found a bug, please report it, as far as possible, with:
 
 - a detailed explanation of steps to reproduce the error
-- a browser and the browser version used
-- a dev tools console exception stack trace (if it is available)
+- the browser and browser version used
+- a dev tools console exception stack trace (if available)
 
-If you found a bug that you consider better discuss in private (for
-example: security bugs), consider first send an email to
+If you found a bug which you think is better to discuss in private (for
+example, security bugs), consider first sending an email to
 `support@penpot.app`.
 
-**We don't have formal bug bounty program for security reports; this
-is an open source application and your contribution will be recognized
+**We don't have a formal bug bounty program for security reports; this
+is an open source application, and your contribution will be recognized
 in the changelog.**
 
 
-## Pull requests ##
+## Pull Requests ##
 
-If you want propose a change or bug fix with the Pull-Request system
-firstly you should carefully read the **DCO** section and format your
-commits accordingly.
+If you want to propose a change or bug fix via a pull request (PR),
+you should first carefully read the section **Developer's Certificate of
+Origin**. You must also format your code and commits according to the
+instructions below.
 
-If you intend to fix a bug it's fine to submit a pull request right
-away but we still recommend to file an issue detailing what you're
+If you intend to fix a bug, it's fine to submit a pull request right
+away, but we still recommend filing an issue detailing what you're
 fixing. This is helpful in case we don't accept that specific fix but
 want to keep track of the issue.
 
-If you want to implement or start working in a new feature, please
-open a **question** / **discussion** issue for it. No pull-request
-will be accepted without previous chat about the changes,
-independently if it is a new feature, already planned feature or small
-quick win.
+If you want to implement or start working on a new feature, please
+open a **question*- / **discussion*- issue for it. No PR
+will be accepted without a prior discussion about the changes,
+whether it is a new feature, an already planned one, or a quick win.
 
-If is going to be your first pull request, You can learn how from this
-free video series:
-
-https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github
-
-We will use the `easy fix` mark for tag for indicate issues that are
-easy for beginners.
+If it is your first PR, you can learn how to proceed from
+[this free video
+series](https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github)
 
+We use the `easy fix` tag to indicate issues that are appropriate for beginners.
 
 ## Commit Guidelines ##
 
-We have very precise rules over how our git commit messages can be formatted.
+We have very precise rules on how our git commit messages must be formatted.
 
 The commit message format is:
 
@@ -71,34 +68,37 @@ The commit message format is:
 Where type is:
 
 - :bug: `:bug:` a commit that fixes a bug
-- :sparkles: `:sparkles:` a commit that an improvement
-- :tada: `:tada:` a commit with new feature
+- :sparkles: `:sparkles:` a commit that adds an improvement
+- :tada: `:tada:` a commit with a new feature
 - :recycle: `:recycle:` a commit that introduces a refactor
 - :lipstick: `:lipstick:` a commit with cosmetic changes
-- :ambulance: `:ambulance:` a commit that fixes critical bug
+- :ambulance: `:ambulance:` a commit that fixes a critical bug
 - :books: `:books:` a commit that improves or adds documentation
-- :construction: `:construction:`: a wip commit
+- :construction: `:construction:` a WIP commit
 - :boom: `:boom:` a commit with breaking changes
 - :wrench: `:wrench:` a commit for config updates
 - :zap: `:zap:` a commit with performance improvements
-- :whale: `:whale:` a commit for docker related stuff
-- :paperclip: `:paperclip:` a commit with other not relevant changes
-- :arrow_up: `:arrow_up:` a commit with dependencies updates
-- :arrow_down: `:arrow_down:` a commit with dependencies downgrades
+- :whale: `:whale:` a commit for Docker-related stuff
+- :paperclip: `:paperclip:` a commit with other non-relevant changes
+- :arrow_up: `:arrow_up:` a commit with dependency updates
+- :arrow_down: `:arrow_down:` a commit with dependency downgrades
 - :fire: `:fire:` a commit that removes files or code
+- :globe_with_meridians: `:globe_with_meridians:` a commit that adds or updates
+  translations
 
 More info:
+
  - https://gist.github.com/parmentf/035de27d6ed1dce0b36a
  - https://gist.github.com/rxaviers/7360908
 
 Each commit should have:
 
-- A concise subject using imperative mood.
-- The subject should have capitalized the first letter, without period
-  at the end and no larger than 65 characters.
+- A concise subject using the imperative mood.
+- The subject should capitalize the first letter, omit the period
+  at the end, and be no longer than 65 characters.
 - A blank line between the subject line and the body.
-- An entry on the CHANGES.md file if applicable, referencing the
-  github or taiga issue/user-story using the these same rules.
+- An entry in the CHANGES.md file if applicable, referencing the
+  GitHub or Taiga issue/user story using these same rules.
 
 Examples of good commit messages:
 
@@ -111,8 +111,30 @@ Examples of good commit messages:
 - `:ambulance: Fix critical bug on user registration process`
 - `:tada: Add new approach for user registration`
 
+## Formatting and Linting ##
 
-## Code of conduct ##
+You will want to make sure your code is formatted and linted before submitting
+a PR. We use [cljfmt](https://github.com/weavejester/cljfmt) and
+[clj-kondo](https://github.com/clj-kondo/clj-kondo) for this. After installing
+them on your system, you can run them with:
+
+```bash
+# Check formatting
+yarn fmt:clj:check
+
+# Check and fix formatting
+yarn fmt:clj
+
+# Run the linter
+yarn lint:clj
+```
+
+There are more choices in `package.json`.
+
+Ideally, you should run these commands as git pre-commit hooks. A convenient way
+of defining them is to use [Husky](https://typicode.github.io/husky/#/).
+
+## Code of Conduct ##
 
 As contributors and maintainers of this project, we pledge to respect
 all people who contribute through reporting issues, posting feature
@@ -132,11 +154,11 @@ unprofessional conduct.
 
 Project maintainers have the right and responsibility to remove, edit,
 or reject comments, commits, code, wiki edits, issues, and other
-contributions that are not aligned to this Code of Conduct. Project
+contributions that are not aligned with this Code of Conduct. Project
 maintainers who do not follow the Code of Conduct may be removed from
 the project team.
 
-This code of conduct applies both within project spaces and in public
+This Code of Conduct applies both within project spaces and in public
 spaces when an individual is representing the project or its
 community.
 
@@ -145,12 +167,11 @@ may be reported by opening an issue or contacting one or more of the
 project maintainers.
 
 This Code of Conduct is adapted from the Contributor Covenant, version
-1.1.0, available from http://contributor-covenant.org/version/1/1/0/
+1.1.0, available from [http://contributor-covenant.org/version/1/1/0/](http://contributor-covenant.org/version/1/1/0/)
 
+## Developer's Certificate of Origin (DCO)
 
-## Developer's Certificate of Origin (DCO) ##
-
-By submitting code you are agree and can certify the below:
+By submitting code you agree to and can certify the following:
 
     Developer's Certificate of Origin 1.1
 
@@ -178,13 +199,15 @@ By submitting code you are agree and can certify the below:
         maintained indefinitely and may be redistributed consistent with
         this project or the open source license(s) involved.
 
-Then, all your code patches (**documentation are excluded**) should
+Then, all your code patches (**documentation is excluded**) should
 contain a sign-off at the end of the patch/commit description body. It
-can be automatically added on adding `-s` parameter to `git commit`.
+can be automatically added by adding the `-s` parameter to `git commit`.
 
-This is an example of the aspect of the line:
+This is an example of what the line should look like:
 
-	Signed-off-by: Andrey Antukh <niwi@niwi.nz>
+```
+Signed-off-by: Andrey Antukh <niwi@niwi.nz>
+```
 
 Please, use your real name (sorry, no pseudonyms or anonymous
 contributions are allowed).

README.md

@@ -16,36 +16,37 @@
 </p>
 
 <p align="center">
-    <a href="https://penpot.app/"><b>Website</b></a>  •  
-    <a href="https://help.penpot.app/technical-guide/getting-started/"><b>Getting Started</b></a>  •  
-    <a href="https://help.penpot.app/user-guide/"><b>User Guide</b></a>  •  
-    <a href="https://help.penpot.app/user-guide/introduction/info/"><b>Tutorials & Info</b></a>  •  
+    <a href="https://penpot.app/"><b>Website</b></a>  •
+    <a href="https://help.penpot.app/user-guide/"><b>User Guide</b></a>  •
+    <a href="https://penpot.app/learning-center"><b>Learning Center</b></a>  •
     <a href="https://community.penpot.app/"><b>Community</b></a>
 </p>
 <p align="center">
-    <a href="https://www.youtube.com/@Penpot"><b>Youtube</b></a>  •  
-    <a href="https://peertube.kaleidos.net/a/penpot_app/video-channels"><b>Peertube</b></a>  •  
-    <a href="https://www.linkedin.com/company/penpot/"><b>Linkedin</b></a>  •  
-    <a href="https://instagram.com/penpot.app"><b>Instagram</b></a>  •  
-    <a href="https://fosstodon.org/@penpot/"><b>Mastodon</b></a>  •  
+    <a href="https://www.youtube.com/@Penpot"><b>Youtube</b></a>  •
+    <a href="https://peertube.kaleidos.net/a/penpot_app/video-channels"><b>Peertube</b></a>  •
+    <a href="https://www.linkedin.com/company/penpot/"><b>Linkedin</b></a>  •
+    <a href="https://instagram.com/penpot.app"><b>Instagram</b></a>  •
+    <a href="https://fosstodon.org/@penpot/"><b>Mastodon</b></a>  •
+    <a href="https://bsky.app/profile/penpot.app"><b>Bluesky</b></a>  •
     <a href="https://twitter.com/penpotapp"><b>X</b></a>
 
 </p>
 
 <br />
 
-[Penpot video](https://github.com/penpot/penpot/assets/5446186/b8ad0764-585e-4ddc-b098-9b4090d337cc)
+[Penpot video](https://github.com/user-attachments/assets/08b83119-c090-4a74-86ed-7bfbdda9a793)
 
 <br />
 
 Penpot is the first **open-source** design tool for design and code collaboration. Designers can create stunning designs, interactive prototypes, design systems at scale, while developers enjoy ready-to-use code and make their workflow easy and fast. And all of this with no handoff drama.
 
-Penpot is available on browser and [self host](https://penpot.app/self-host). It’s web-based and works with open standards (SVG, CSS and HTML). And last but not least, it’s free! 
+Available on browser or self-hosted, Penpot works with open standards like SVG, CSS, HTML and JSON, and it’s free!
 
-Penpot’s latest [huge release 2.0](https://penpot.app/dev-diaries), takes the platform to a whole new level. This update introduces the ground-breaking [CSS Grid Layout feature](https://penpot.app/penpot-2.0), a complete UI redesign, a new Components system, and much more. Plus, it's faster and more accessible. 
+The latest updates take Penpot even further. It’s the first design tool to integrate native [design tokens](https://penpot.dev/collaboration/design-tokens)—a single source of truth to improve efficiency and collaboration between product design and development.
+With the [huge 2.0 release](https://penpot.app/dev-diaries), Penpot took the platform to a whole new level. This update introduces the ground-breaking [CSS Grid Layout feature](https://penpot.app/penpot-2.0), a complete UI redesign, a new Components system, and much more.
+For organizations that need extra service for its teams, [get in touch](https://cal.com/team/penpot/talk-to-us)
 
-
-🎇 **Penpot Fest** is our design, code & Open Source event. Check out the highlights from [Penpot Fest 2023 edition](https://www.youtube.com/watch?v=sOpLZaK5mDc)!
+🎇 Design, code, and Open Source meet at [Penpot Fest](https://penpot.app/penpotfest)! Be part of the 2025 edition in Madrid, Spain, on October 9-10.
 
 ## Table of contents ##
 
@@ -61,7 +62,7 @@ Penpot’s latest [huge release 2.0](https://penpot.app/dev-diaries), takes the
 Penpot expresses designs as code. Designers can do their best work and see it will be beautifully implemented by developers in a two-way collaboration.
 
 ### Plugin system ###
-[Penpot plugins](https://penpot.app/penpothub/plugins) let you expand the platform's capabilities, give you the flexibility to integrate it with other apps, and design custom solutions. 
+[Penpot plugins](https://penpot.app/penpothub/plugins) let you expand the platform's capabilities, give you the flexibility to integrate it with other apps, and design custom solutions.
 
 ### Designed for developers ###
 Penpot was built to serve both designers and developers and create a fluid design-code process. You have the choice to enjoy real-time collaboration or play "solo".
@@ -78,6 +79,10 @@ Penpot offers integration into the development toolchain, thanks to its support
 ### What’s great for design ###
 With Penpot you can design libraries to share and reuse; turn design elements into components and tokens to allow reusability and scalability; and build realistic user flows and interactions.
 
+### Design Tokens ###
+With Penpot’s standardized [design tokens](https://penpot.dev/collaboration/design-tokens) format, you can easily reuse and sync tokens across different platforms, workflows, and disciplines.
+
+
 <br />
 
 <p align="center">
@@ -125,13 +130,13 @@ You will find the following categories:
 
 ## Contributing ##
 
-Any contribution will make a difference to improve Penpot. How can you get involved? 
+Any contribution will make a difference to improve Penpot. How can you get involved?
 
-Choose your way: 
+Choose your way:
 
 - Create and [share Libraries & Templates](https://penpot.app/libraries-templates.html) that will be helpful for the community
 - Invite your [team to join](https://design.penpot.app/#/auth/register)
-- Star this repo and follow us on Social Media: [Mastodon](https://fosstodon.org/@penpot/), [Youtube](https://www.youtube.com/c/Penpot), [Instagram](https://instagram.com/penpot.app), [Linkedin](https://www.linkedin.com/company/penpotdesign),  [Peertube](https://peertube.kaleidos.net/a/penpot_app) and [X](https://twitter.com/penpotapp).
+- Give this repo a star and follow us on Social Media: [Mastodon](https://fosstodon.org/@penpot/), [Youtube](https://www.youtube.com/c/Penpot), [Instagram](https://instagram.com/penpot.app), [Linkedin](https://www.linkedin.com/company/penpotdesign),  [Peertube](https://peertube.kaleidos.net/a/penpot_app), [X](https://twitter.com/penpotapp) and [BlueSky](https://bsky.app/profile/penpot.app)
 - Participate in the [Community](https://community.penpot.app/) space by asking and answering questions; reacting to others’ articles;  opening your own conversations and following along on decisions affecting the project.
 - Report bugs with our easy [guide for bugs hunting](https://help.penpot.app/contributing-guide/reporting-bugs/) or [GitHub issues](https://github.com/penpot/penpot/issues)
 - Become a [translator](https://help.penpot.app/contributing-guide/translations)

backend/deps.edn

@@ -6,7 +6,7 @@
   org.clojure/clojure {:mvn/version "1.12.0"}
   org.clojure/tools.namespace {:mvn/version "1.5.0"}
 
-  com.github.luben/zstd-jni {:mvn/version "1.5.6-9"}
+  com.github.luben/zstd-jni {:mvn/version "1.5.7-3"}
 
   io.prometheus/simpleclient {:mvn/version "0.16.0"}
   io.prometheus/simpleclient_hotspot {:mvn/version "0.16.0"}
@@ -17,7 +17,7 @@
 
   io.prometheus/simpleclient_httpserver {:mvn/version "0.16.0"}
 
-  io.lettuce/lettuce-core {:mvn/version "6.5.2.RELEASE"}
+  io.lettuce/lettuce-core {:mvn/version "6.6.0.RELEASE"}
   java-http-clj/java-http-clj {:mvn/version "0.4.3"}
 
   funcool/yetti
@@ -27,15 +27,15 @@
    :exclusions [org.slf4j/slf4j-api]}
 
   com.github.seancorfield/next.jdbc
-  {:mvn/version "1.3.994"}
-  metosin/reitit-core {:mvn/version "0.7.2"}
+  {:mvn/version "1.3.1002"}
+  metosin/reitit-core {:mvn/version "0.8.0"}
   nrepl/nrepl {:mvn/version "1.3.1"}
-  cider/cider-nrepl {:mvn/version "0.52.0"}
+  cider/cider-nrepl {:mvn/version "0.55.7"}
 
   org.postgresql/postgresql {:mvn/version "42.7.5"}
-  org.xerial/sqlite-jdbc {:mvn/version "3.48.0.0"}
+  org.xerial/sqlite-jdbc {:mvn/version "3.49.1.0"}
 
-  com.zaxxer/HikariCP {:mvn/version "6.2.1"}
+  com.zaxxer/HikariCP {:mvn/version "6.3.0"}
 
   io.whitfin/siphash {:mvn/version "2.0.0"}
 
@@ -44,7 +44,7 @@
 
   com.github.ben-manes.caffeine/caffeine {:mvn/version "3.2.0"}
 
-  org.jsoup/jsoup {:mvn/version "1.18.3"}
+  org.jsoup/jsoup {:mvn/version "1.20.1"}
   org.im4java/im4java
   {:git/tag "1.4.0-penpot-2"
    :git/sha "e2b3e16"
@@ -55,11 +55,11 @@
   org.clojars.pntblnk/clj-ldap {:mvn/version "0.0.17"}
 
   dawran6/emoji {:mvn/version "0.1.5"}
-  markdown-clj/markdown-clj {:mvn/version "1.12.2"}
+  markdown-clj/markdown-clj {:mvn/version "1.12.3"}
 
   ;; Pretty Print specs
   pretty-spec/pretty-spec {:mvn/version "0.1.4"}
-  software.amazon.awssdk/s3 {:mvn/version "2.28.26"}}
+  software.amazon.awssdk/s3 {:mvn/version "2.31.48"}}
 
  :paths ["src" "resources" "target/classes"]
  :aliases
@@ -74,7 +74,7 @@
 
   :build
   {:extra-deps
-   {io.github.clojure/tools.build {:git/tag "v0.10.6" :git/sha "52cf7d6"}}
+   {io.github.clojure/tools.build {:git/tag "v0.10.9" :git/sha "e405aac"}}
    :ns-default build}
 
   :test

backend/package.json

@@ -4,7 +4,7 @@
   "license": "MPL-2.0",
   "author": "Kaleidos INC",
   "private": true,
-  "packageManager": "yarn@4.6.0+sha512.5383cc12567a95f1d668fbe762dfe0075c595b4bfff433be478dbbe24e05251a8e8c3eb992a986667c1d53b6c3a9c85b8398c35a960587fbd9fa3a0915406728",
+  "packageManager": "yarn@4.9.1+sha512.f95ce356460e05be48d66401c1ae64ef84d163dd689964962c6888a9810865e39097a5e9de748876c2e0bf89b232d583c33982773e9903ae7a76257270986538",
   "repository": {
     "type": "git",
     "url": "https://github.com/penpot/penpot"

backend/resources/app/templates/debug.tmpl

@@ -151,6 +151,78 @@ Debug Main Page
         </div>
 
 
+        <div class="row">
+          <input type="submit" value="Submit" />
+        </div>
+      </form>
+    </fieldset>
+  </section>
+
+  <section class="widget">
+    <h2>Feature Flags</h2>
+    <fieldset>
+      <legend>Enable</legend>
+      <desc>Add a feature flag to a team</desc>
+      <form method="post" action="/dbg/actions/add-team-feature">
+        <div class="row">
+          <input type="text" style="width:300px" name="team-id" placeholder="team-id" />
+        </div>
+        <div class="row">
+          <input type="text" style="width:100px" name="feature" placeholder="feature" value="" />
+        </div>
+
+        <div class="row">
+          <label for="check-feature">Skip feature check</label>
+          <input id="check-feature" type="checkbox" name="skip-check" />
+          <br />
+          <small>
+            Do not check if the feature is supported
+          </small>
+        </div>
+
+        <div class="row">
+          <label for="force-version">Are you sure?</label>
+          <input id="force-version" type="checkbox" name="force" />
+          <br />
+          <small>
+            This is a just a security double check for prevent non intentional submits.
+          </small>
+        </div>
+
+        <div class="row">
+          <input type="submit" value="Submit" />
+        </div>
+      </form>
+    </fieldset>
+    <fieldset>
+      <legend>Disable</legend>
+      <desc>Remove a feature flag from a team</desc>
+      <form method="post" action="/dbg/actions/remove-team-feature">
+        <div class="row">
+          <input type="text" style="width:300px" name="team-id" placeholder="team-id" />
+        </div>
+        <div class="row">
+          <input type="text" style="width:100px" name="feature" placeholder="feature" value="" />
+        </div>
+
+        <div class="row">
+          <label for="check-feature">Skip feature check</label>
+          <input id="check-feature" type="checkbox" name="skip-check" />
+          <br />
+          <small>
+            Do not check if the feature is supported
+          </small>
+        </div>
+
+        <div class="row">
+          <label for="force-version">Are you sure?</label>
+          <input id="force-version" type="checkbox" name="force" />
+          <br />
+          <small>
+            This is a just a security double check for prevent non intentional submits.
+          </small>
+        </div>
+
         <div class="row">
           <input type="submit" value="Submit" />
         </div>

backend/scripts/manage.py

@@ -35,40 +35,35 @@ def get_prepl_conninfo():
 
     return host, port
 
-def send_eval(expr):
+def send(data):
     host, port = get_prepl_conninfo()
+    with socket.create_connection((host, port)) as s:
+        f = s.makefile(mode="rw")
 
-    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
-        s.connect((host, port))
-        s.send(expr.encode("utf-8"))
-        s.send(b":repl/quit\n\n")
+        json.dump(data, f)
+        f.write("\n")
+        f.flush()
 
-        with s.makefile() as f:
-            while True:
-                line = f.readline()
-                result = json.loads(line)
-                tag = result.get("tag", None)
-                if tag == "ret":
-                    return result.get("val", None), result.get("exception", None)
-                elif tag == "out":
-                    print(result.get("val"), end="")
-                else:
-                    raise RuntimeError("unexpected response from PREPL")
+        while True:
+            line = f.readline()
+            result = json.loads(line)
+            tag = result.get("tag", None)
 
-def encode(val):
-    return json.dumps(json.dumps(val))
+            if tag == "ret":
+                return result.get("val", None), result.get("err", None)
+            elif tag == "out":
+                print(result.get("val"), end="")
+            else:
+                raise RuntimeError("unexpected response from PREPL")
 
-def print_error(res):
-    for error in res["via"]:
-        print("ERR:", error["message"])
-        break
+def print_error(error):
+    print("ERR:", error["hint"])
 
 def run_cmd(params):
     try:
-        expr = "(app.srepl.cli/exec {})".format(encode(params))
-        res, failed = send_eval(expr)
-        if failed:
-            print_error(res)
+        res, err = send(params)
+        if err:
+            print_error(err)
             sys.exit(-1)
 
         return res
@@ -96,7 +91,7 @@ def update_profile(email, fullname, password, is_active):
             "email": email,
             "fullname": fullname,
             "password": password,
-            "is_active": is_active
+            "isActive": is_active
         }
     }
 
@@ -138,7 +133,7 @@ def derive_password(password):
     params = {
         "cmd": "derive-password",
         "params": {
-            "password": password,
+            "password": password
         }
     }
 

backend/scripts/repl

@@ -30,7 +30,9 @@ export PENPOT_FLAGS="\
        enable-access-tokens \
        enable-tiered-file-data-storage \
        enable-file-validation \
-       enable-file-schema-validation";
+       enable-file-schema-validation \
+       enable-subscriptions \
+       enable-subscriptions-old";
 
 # Default deletion delay for devenv
 export PENPOT_DELETION_DELAY="24h"
@@ -70,15 +72,18 @@ export PENPOT_OBJECTS_STORAGE_S3_ENDPOINT=http://minio:9000
 export PENPOT_OBJECTS_STORAGE_S3_BUCKET=penpot
 export PENPOT_OBJECTS_STORAGE_FS_DIRECTORY="assets"
 
-export JAVA_OPTS="--enable-preview \
+export JAVA_OPTS="\
        -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager \
        -Djdk.attach.allowAttachSelf \
        -Dlog4j2.configurationFile=log4j2-devenv-repl.xml \
        -Djdk.tracePinnedThreads=full \
        -XX:+EnableDynamicAgentLoading \
-       -XX:-OmitStackTraceInFastThrow \
+       -XX:-OmitStackTraceInFastThrow                    \
        -XX:+UnlockDiagnosticVMOptions \
-       -XX:+DebugNonSafepoints";
+       -XX:+DebugNonSafepoints \
+       --sun-misc-unsafe-memory-access=allow \
+       --enable-preview \
+       --enable-native-access=ALL-UNNAMED";
 
 export OPTIONS="-A:jmx-remote -A:dev"
 

backend/scripts/run.template.sh

@@ -18,7 +18,7 @@ if [ -f ./environ ]; then
     source ./environ
 fi
 
-export JVM_OPTS="-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dlog4j2.configurationFile=log4j2.xml -XX:-OmitStackTraceInFastThrow --enable-preview $JVM_OPTS"
+export JVM_OPTS="-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dlog4j2.configurationFile=log4j2.xml -XX:-OmitStackTraceInFastThrow --enable-native-access=ALL-UNNAMED --enable-preview $JVM_OPTS"
 
 ENTRYPOINT=${1:-app.main};
 

backend/scripts/start-dev

@@ -23,18 +23,9 @@ export PENPOT_FLAGS="\
        enable-access-tokens \
        enable-tiered-file-data-storage \
        enable-file-validation \
-       enable-file-schema-validation";
-
-export OPTIONS="
-       -A:jmx-remote -A:dev \
-       -J-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager \
-       -J-Djdk.attach.allowAttachSelf \
-       -J-Dpolyglot.engine.WarnInterpreterOnly=false \
-       -J-Dlog4j2.configurationFile=log4j2-devenv.xml \
-       -J-XX:+EnableDynamicAgentLoading \
-       -J-XX:-OmitStackTraceInFastThrow \
-       -J-XX:+UnlockDiagnosticVMOptions \
-       -J-XX:+DebugNonSafepoints"
+       enable-file-schema-validation \
+       enable-subscriptions \
+       enable-subscriptions-old ";
 
 # Default deletion delay for devenv
 export PENPOT_DELETION_DELAY="24h"
@@ -65,6 +56,20 @@ export PENPOT_OBJECTS_STORAGE_S3_BUCKET=penpot
 
 entrypoint=${1:-app.main};
 
-set -ex
+export JAVA_OPTS="\
+       -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager \
+       -Djdk.attach.allowAttachSelf \
+       -Dlog4j2.configurationFile=log4j2-devenv.xml \
+       -Djdk.tracePinnedThreads=full \
+       -XX:+EnableDynamicAgentLoading \
+       -XX:-OmitStackTraceInFastThrow                    \
+       -XX:+UnlockDiagnosticVMOptions \
+       -XX:+DebugNonSafepoints \
+       --sun-misc-unsafe-memory-access=allow \
+       --enable-preview \
+       --enable-native-access=ALL-UNNAMED";
 
-clojure $OPTIONS -A:dev -M -m $entrypoint;
+export OPTIONS="-A:jmx-remote -A:dev"
+
+set -ex
+clojure $OPTIONS -M -m $entrypoint;

backend/src/app/binfile/cleaner.clj

@@ -0,0 +1,107 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.binfile.cleaner
+  "A collection of helpers for perform cleaning of artifacts; mainly
+  for recently imported shapes."
+  (:require
+   [app.common.data :as d]
+   [app.common.types.shape :as cts]
+   [app.common.uuid :as uuid]))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; PRE DECODE
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn clean-shape-pre-decode
+  "Applies a pre-decode phase migration to the shape"
+  [shape]
+  (if (= "bool" (:type shape))
+    (if-let [content (get shape :bool-content)]
+      (-> shape
+          (assoc :content content)
+          (dissoc :bool-content))
+      shape)
+    shape))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; POST DECODE
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn- fix-shape-shadow-color
+  "Some shapes can come with invalid `id` property on shadow colors
+  caused by incorrect uuid parsing bug that should be already fixed;
+  this function removes the invalid id from the data structure."
+  [shape]
+  (let [fix-color
+        (fn [{:keys [id] :as color}]
+          (if (uuid? id)
+            color
+            (if (and (string? id)
+                     (re-matches uuid/regex id))
+              (assoc color :id (uuid/uuid id))
+              (dissoc color :id))))
+
+        fix-shadow
+        (fn [shadow]
+          (d/update-when shadow :color fix-color))
+
+        xform
+        (map fix-shadow)]
+
+    (d/update-when shape :shadow
+                   (fn [shadows]
+                     (into [] xform shadows)))))
+
+(defn- fix-root-shape
+  "Ensure all root objects are well formed shapes"
+  [shape]
+  (if (= (:id shape) uuid/zero)
+    (-> shape
+        (assoc :parent-id uuid/zero)
+        (assoc :frame-id uuid/zero)
+        ;; We explicitly dissoc them and let the shape-setup
+        ;; to regenerate it with valid values.
+        (dissoc :selrect)
+        (dissoc :points)
+        (cts/setup-shape))
+    shape))
+
+(defn- fix-legacy-flex-dir
+  "This operation is only relevant to old data and it is fixed just
+  for convenience."
+  [shape]
+  (d/update-when shape :layout-flex-dir
+                 (fn [dir]
+                   (case dir
+                     :reverse-row :row-reverse
+                     :reverse-column :column-reverse
+                     dir))))
+
+(defn clean-shape-post-decode
+  "A shape procesor that expected to be executed after schema decoding
+  process but before validation."
+  [shape]
+  (-> shape
+      (fix-shape-shadow-color)
+      (fix-root-shape)
+      (fix-legacy-flex-dir)))
+
+(defn- fix-container
+  [container]
+  (-> container
+      ;; Remove possible `nil` keys on objects
+      (d/update-when :objects dissoc nil)
+      (d/update-when :objects d/update-vals clean-shape-post-decode)))
+
+(defn clean-file
+  [file & {:as _opts}]
+  (update file :data
+          (fn [data]
+            (-> data
+                (d/update-when :pages-index d/update-vals fix-container)
+                (d/update-when :components d/update-vals fix-container)
+                (d/without-nils)))))

backend/src/app/binfile/common.clj

@@ -431,15 +431,21 @@
                           (update :components relink-shapes)
                           (update :media relink-media)
                           (update :colors relink-colors)
-                          (d/without-nils))))))
+                          (d/without-nils))))
+
+      ;; NOTE: this is necessary because when we just creating a new
+      ;; file from imported artifact or cloned file there are no
+      ;; migrations registered on the database, so we need to persist
+      ;; all of them, not only the applied
+      (vary-meta dissoc ::fmg/migrated)))
 
 (defn encode-file
-  [{:keys [::db/conn] :as cfg} {:keys [id] :as file}]
-  (let [file (if (contains? (:features file) "fdata/objects-map")
+  [{:keys [::db/conn] :as cfg} {:keys [id features] :as file}]
+  (let [file (if (contains? features "fdata/objects-map")
                (feat.fdata/enable-objects-map file)
                file)
 
-        file (if (contains? (:features file) "fdata/pointer-map")
+        file (if (contains? features "fdata/pointer-map")
                (binding [pmap/*tracked* (pmap/create-tracked)]
                  (let [file (feat.fdata/enable-pointer-map file)]
                    (feat.fdata/persist-pointers! cfg id)

backend/src/app/binfile/migrations.clj

@@ -10,7 +10,6 @@
    [app.binfile.common :as bfc]
    [app.common.exceptions :as ex]
    [app.common.features :as cfeat]
-   [app.features.components-v2 :as feat.compv2]
    [clojure.set :as set]
    [cuerdas.core :as str]))
 
@@ -28,17 +27,20 @@
 
 (defn apply-pending-migrations!
   "Apply alredy registered pending migrations to files"
-  [cfg]
-  (doseq [[feature file-id] (-> bfc/*state* deref :pending-to-migrate)]
+  [_cfg]
+  (doseq [[feature _file-id] (-> bfc/*state* deref :pending-to-migrate)]
     (case feature
       "components/v2"
-      (feat.compv2/migrate-file! cfg file-id
-                                 :validate? (::validate cfg true)
-                                 :skip-on-graphic-error? true)
+      nil
 
       "fdata/shape-data-type"
       nil
 
+      ;; There is no migration needed, but we don't want to allow
+      ;; copy paste nor import of variant files into no-variant teams
+      "variants/v1"
+      nil
+
       (ex/raise :type :internal
                 :code :no-migration-defined
                 :hint (str/ffmt "no migation for feature '%' on file importation" feature)

backend/src/app/binfile/v3.clj

@@ -8,14 +8,17 @@
   "A ZIP based binary file exportation"
   (:refer-clojure :exclude [read])
   (:require
+   [app.binfile.cleaner :as bfl]
    [app.binfile.common :as bfc]
    [app.binfile.migrations :as bfm]
    [app.common.data :as d]
    [app.common.data.macros :as dm]
    [app.common.exceptions :as ex]
    [app.common.features :as cfeat]
+   [app.common.files.migrations :as-alias fmg]
    [app.common.json :as json]
    [app.common.logging :as l]
+   [app.common.media :as cmedia]
    [app.common.schema :as sm]
    [app.common.thumbnails :as cth]
    [app.common.types.color :as ctcl]
@@ -71,7 +74,7 @@
    [:size ::sm/int]
    [:content-type :string]
    [:bucket [::sm/one-of {:format :string} sto/valid-buckets]]
-   [:hash :string]])
+   [:hash {:optional true} :string]])
 
 (def ^:private schema:file-thumbnail
   [:map {:title "FileThumbnail"}
@@ -86,13 +89,19 @@
    ctf/schema:file
    [:map [:options {:optional true} ctf/schema:options]]])
 
+;; --- HELPERS
+
+(defn- default-now
+  [o]
+  (or o (dt/now)))
+
 ;; --- ENCODERS
 
 (def encode-file
   (sm/encoder schema:file sm/json-transformer))
 
 (def encode-page
-  (sm/encoder ::ctp/page sm/json-transformer))
+  (sm/encoder ctp/schema:page sm/json-transformer))
 
 (def encode-shape
   (sm/encoder ::cts/shape sm/json-transformer))
@@ -127,7 +136,7 @@
   (sm/decoder schema:manifest sm/json-transformer))
 
 (def decode-media
-  (sm/decoder ::ctf/media sm/json-transformer))
+  (sm/decoder ctf/schema:media sm/json-transformer))
 
 (def decode-component
   (sm/decoder ::ctc/component sm/json-transformer))
@@ -227,27 +236,13 @@
                         :always
                         (bfc/clean-file-features))))))
 
-(defn- resolve-extension
-  [mtype]
-  (case mtype
-    "image/png"     ".png"
-    "image/jpeg"    ".jpg"
-    "image/gif"     ".gif"
-    "image/svg+xml" ".svg"
-    "image/webp"    ".webp"
-    "font/woff"     ".woff"
-    "font/woff2"    ".woff2"
-    "font/ttf"      ".ttf"
-    "font/otf"      ".otf"
-    "application/octet-stream" ".bin"))
-
 (defn- export-storage-objects
   [{:keys [::output] :as cfg}]
   (let [storage (sto/resolve cfg)]
     (doseq [id (-> bfc/*state* deref :storage-objects not-empty)]
       (let [sobject (sto/get-object storage id)
             smeta   (meta sobject)
-            ext     (resolve-extension (:content-type smeta))
+            ext     (cmedia/mtype->extension (:content-type smeta))
             path    (str "objects/" id ".json")
             params  (-> (meta sobject)
                         (assoc :id (:id sobject))
@@ -572,7 +567,13 @@
                  (let [object (->> (read-entry input entry)
                                    (decode-media)
                                    (validate-media))
-                       object (assoc object :file-id file-id)]
+                       object (-> object
+                                  (assoc :file-id file-id)
+                                  (update :created-at default-now)
+                                  ;; FIXME: this is set default to true for
+                                  ;; setting a value, this prop is no longer
+                                  ;; relevant;
+                                  (assoc :is-local true))]
                    (if (= id (:id object))
                      (conj result object)
                      result)))
@@ -594,16 +595,35 @@
 
 (defn- read-file-components
   [{:keys [::bfc/input ::file-id ::entries]}]
-  (->> (keep (match-component-entry-fn file-id) entries)
-       (reduce (fn [result {:keys [id entry]}]
-                 (let [object (->> (read-entry input entry)
-                                   (decode-component)
-                                   (validate-component))]
-                   (if (= id (:id object))
-                     (assoc result id object)
-                     result)))
-               {})
-       (not-empty)))
+  (let [clean-component-post-decode
+        (fn [component]
+          (d/update-when component :objects
+                         (fn [objects]
+                           (reduce-kv (fn [objects id shape]
+                                        (assoc objects id (bfl/clean-shape-post-decode shape)))
+                                      objects
+                                      objects))))
+        clean-component-pre-decode
+        (fn [component]
+          (d/update-when component :objects
+                         (fn [objects]
+                           (reduce-kv (fn [objects id shape]
+                                        (assoc objects id (bfl/clean-shape-pre-decode shape)))
+                                      objects
+                                      objects))))]
+
+    (->> (keep (match-component-entry-fn file-id) entries)
+         (reduce (fn [result {:keys [id entry]}]
+                   (let [object (->> (read-entry input entry)
+                                     (clean-component-pre-decode)
+                                     (decode-component)
+                                     (clean-component-post-decode)
+                                     (validate-component))]
+                     (if (= id (:id object))
+                       (assoc result id object)
+                       result)))
+                 {})
+         (not-empty))))
 
 (defn- read-file-typographies
   [{:keys [::bfc/input ::file-id ::entries]}]
@@ -630,7 +650,9 @@
   (->> (keep (match-shape-entry-fn file-id page-id) entries)
        (reduce (fn [result {:keys [id entry]}]
                  (let [object (->> (read-entry input entry)
+                                   (bfl/clean-shape-pre-decode)
                                    (decode-shape)
+                                   (bfl/clean-shape-post-decode)
                                    (validate-shape))]
                    (if (= id (:id object))
                      (assoc result id object)
@@ -735,7 +757,6 @@
                    (dissoc :options)
                    (bfc/process-file))]
 
-
       (bfm/register-pending-migrations! cfg file)
       (bfc/save-file! cfg file ::db/return-keys false)
 
@@ -778,7 +799,7 @@
                     :expected-id (str id)
                     :found-id (str (:id object))))
 
-        (let [ext     (resolve-extension (:content-type object))
+        (let [ext     (cmedia/mtype->extension (:content-type object))
               path    (str "objects/" id ext)
               content (->> path
                            (get-zip-entry input)
@@ -792,13 +813,14 @@
                       :expected-size (:size object)
                       :found-size (sto/get-size content)))
 
-          (when (not= (:hash object) (sto/get-hash content))
-            (ex/raise :type :validation
-                      :code :inconsistent-penpot-file
-                      :hint "found corrupted storage object: hash does not match"
-                      :path path
-                      :expected-hash (:hash object)
-                      :found-hash (sto/get-hash content)))
+          (when-let [hash (get object :hash)]
+            (when (not= hash (sto/get-hash content))
+              (ex/raise :type :validation
+                        :code :inconsistent-penpot-file
+                        :hint "found corrupted storage object: hash does not match"
+                        :path path
+                        :expected-hash (:hash object)
+                        :found-hash (sto/get-hash content))))
 
           (let [params  (-> object
                             (dissoc :id :size)

backend/src/app/db.clj

@@ -22,7 +22,8 @@
    [clojure.set :as set]
    [integrant.core :as ig]
    [next.jdbc :as jdbc]
-   [next.jdbc.date-time :as jdbc-dt])
+   [next.jdbc.date-time :as jdbc-dt]
+   [next.jdbc.transaction])
   (:import
    com.zaxxer.hikari.HikariConfig
    com.zaxxer.hikari.HikariDataSource
@@ -41,6 +42,8 @@
    org.postgresql.util.PGInterval
    org.postgresql.util.PGobject))
 
+(def ^:dynamic *conn* nil)
+
 (declare open)
 (declare create-pool)
 
@@ -223,16 +226,6 @@
     (let [^OutputStream os (.getOutputStream ^LargeObject lobj)]
       (io/make-output-stream os opts))))
 
-(defmacro with-atomic
-  [& args]
-  (if (symbol? (first args))
-    (let [cfgs (first args)
-          body (rest args)]
-      `(jdbc/with-transaction [conn# (::pool ~cfgs)]
-         (let [~cfgs (assoc ~cfgs ::conn conn#)]
-           ~@body)))
-    `(jdbc/with-transaction ~@args)))
-
 (defn open
   [system-or-pool]
   (if (pool? system-or-pool)
@@ -535,43 +528,31 @@
      (l/trc :hint "explicit rollback requested (savepoint)")
      (.rollback conn sp))))
 
+(defn transact!
+  "A lower-level function for executing function in a transaction"
+  ([transactable f] (transact! transactable f {}))
+  ([transactable f opts]
+   (binding [next.jdbc.transaction/*nested-tx* :ignore]
+     (jdbc/transact transactable f opts))))
+
 (defn tx-run!
+  "Run a function in a transaction."
   [system f & params]
-  (cond
-    (connection? system)
+  (if (connection? system)
     (tx-run! {::conn system} f)
-
-    (pool? system)
-    (tx-run! {::pool system} f)
-
-    (::conn system)
-    (let [conn (::conn system)
-          sp   (savepoint conn)]
-      (try
-        (let [system' (-> system
-                          (assoc ::savepoint sp)
-                          (dissoc ::rollback))
-              result  (apply f system' params)]
-          (if (::rollback system)
-            (rollback! conn sp)
-            (release! conn sp))
-          result)
-        (catch Throwable cause
-          (.rollback ^Connection conn ^Savepoint sp)
-          (throw cause))))
-
-    (::pool system)
-    (with-atomic [conn (::pool system)]
-      (let [system' (-> system
-                        (assoc ::conn conn)
-                        (dissoc ::rollback))
-            result  (apply f system' params)]
-        (when (::rollback system)
-          (rollback! conn))
-        result))
-
-    :else
-    (throw (IllegalArgumentException. "invalid system/cfg provided"))))
+    (if (pool? system)
+      (tx-run! {::pool system} f)
+      (if-let [conn (or (::conn system)
+                        (::pool system))]
+        (transact! conn
+                   (fn [conn]
+                     (let [system' (-> system
+                                       (dissoc ::rollback)
+                                       (assoc ::conn conn))]
+                       (apply f system' params)))
+                   {:rollback-only (::rollback system)
+                    :read-only (::read-only system)})
+        (throw (IllegalArgumentException. "invalid system/cfg provided"))))))
 
 (defn run!
   [system f & params]

backend/src/app/features/fdata.clj

@@ -9,7 +9,10 @@
   (:require
    [app.common.data :as d]
    [app.common.exceptions :as ex]
+   [app.common.files.helpers :as cfh]
+   [app.common.files.migrations :as fmg]
    [app.common.logging :as l]
+   [app.common.types.path :as path]
    [app.db :as db]
    [app.db.sql :as-alias sql]
    [app.storage :as sto]
@@ -30,7 +33,7 @@
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
 (defn enable-objects-map
-  [file]
+  [file & _opts]
   (let [update-page
         (fn [page]
           (if (and (pmap/pointer-map? page)
@@ -136,10 +139,56 @@
 
 (defn enable-pointer-map
   "Enable the fdata/pointer-map feature on the file."
-  [file]
+  [file & _opts]
   (-> file
       (update :data (fn [fdata]
                       (-> fdata
                           (update :pages-index d/update-vals pmap/wrap)
                           (d/update-when :components pmap/wrap))))
       (update :features conj "fdata/pointer-map")))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; PATH-DATA
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn enable-path-data
+  "Enable the fdata/path-data feature on the file."
+  [file & _opts]
+  (letfn [(update-object [object]
+            (if (or (cfh/path-shape? object)
+                    (cfh/bool-shape? object))
+              (update object :content path/content)
+              object))
+
+          (update-container [container]
+            (d/update-when container :objects d/update-vals update-object))]
+
+    (-> file
+        (update :data (fn [data]
+                        (-> data
+                            (update :pages-index d/update-vals update-container)
+                            (d/update-when :components d/update-vals update-container))))
+        (update :features conj "fdata/path-data"))))
+
+(defn disable-path-data
+  [file & _opts]
+  (letfn [(update-object [object]
+            (if (or (cfh/path-shape? object)
+                    (cfh/bool-shape? object))
+              (update object :content vec)
+              object))
+
+          (update-container [container]
+            (d/update-when container :objects d/update-vals update-object))]
+
+    (when-let [conn db/*conn*]
+      (db/delete! conn :file-migration {:file-id (:id file)
+                                        :name "0003-convert-path-content"}))
+    (-> file
+        (update :data (fn [data]
+                        (-> data
+                            (update :pages-index d/update-vals update-container)
+                            (d/update-when :components d/update-vals update-container))))
+        (update :features disj "fdata/path-data")
+        (update :migrations disj "0003-convert-path-content")
+        (vary-meta update ::fmg/migrated disj "0003-convert-path-content"))))

backend/src/app/http.clj

@@ -155,10 +155,10 @@
    [["" {:middleware [[mw/server-timing]
                       [mw/params]
                       [mw/format-response]
-                      [mw/parse-request]
-                      [mw/errors errors/handle]
                       [session/soft-auth cfg]
                       [actoken/soft-auth cfg]
+                      [mw/parse-request]
+                      [mw/errors errors/handle]
                       [mw/restrict-methods]]}
 
      (::mtx/routes cfg)

backend/src/app/http/debug.clj

@@ -24,7 +24,7 @@
    [app.rpc.commands.profile :as profile]
    [app.rpc.commands.teams :as teams]
    [app.setup :as-alias setup]
-   [app.srepl.helpers :as srepl]
+   [app.srepl.main :as srepl]
    [app.storage :as-alias sto]
    [app.storage.tmp :as tmp]
    [app.util.blob :as blob]
@@ -430,6 +430,50 @@
      ::yres/body    "OK"}))
 
 
+(defn- add-team-feature
+  [{:keys [params] :as request}]
+  (let [team-id (some-> params :team-id d/parse-uuid)
+        feature (some-> params :feature str)
+        skip-check (contains? params :skip-check)]
+
+    (when-not (contains? params :force)
+      (ex/raise :type :validation
+                :code :missing-force
+                :hint "missing force checkbox"))
+
+    (when (nil? team-id)
+      (ex/raise :type :validation
+                :code :invalid-team-id
+                :hint "provided invalid team id"))
+
+    (srepl/enable-team-feature! team-id feature :skip-check skip-check)
+
+    {::yres/status  200
+     ::yres/headers {"content-type" "text/plain"}
+     ::yres/body    "OK"}))
+
+(defn- remove-team-feature
+  [{:keys [params] :as request}]
+  (let [team-id   (some-> params :team-id d/parse-uuid)
+        feature   (some-> params :feature str)
+        skip-check (contains? params :skip-check)]
+
+    (when-not (contains? params :force)
+      (ex/raise :type :validation
+                :code :missing-force
+                :hint "missing force checkbox"))
+
+    (when (nil? team-id)
+      (ex/raise :type :validation
+                :code :invalid-team-id
+                :hint "provided invalid team id"))
+
+    (srepl/disable-team-feature! team-id feature :skip-check skip-check)
+
+    {::yres/status  200
+     ::yres/headers {"content-type" "text/plain"}
+     ::yres/body    "OK"}))
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; OTHER SMALL VIEWS/HANDLERS
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
@@ -500,6 +544,10 @@
      {:handler (partial resend-email-notification cfg)}]
     ["/actions/reset-file-version"
      {:handler (partial reset-file-version cfg)}]
+    ["/actions/add-team-feature"
+     {:handler (partial add-team-feature)}]
+    ["/actions/remove-team-feature"
+     {:handler (partial remove-team-feature)}]
     ["/file/export" {:handler (partial export-handler cfg)}]
     ["/file/import" {:handler (partial import-handler cfg)}]
     ["/file/data" {:handler (partial file-data-handler cfg)}]

backend/src/app/http/errors.clj

@@ -25,7 +25,6 @@
   (let [claims (-> {}
                    (into (::session/token-claims request))
                    (into (::actoken/token-claims request)))]
-
     {:request/path       (:path request)
      :request/method     (:method request)
      :request/params     (:params request)
@@ -62,7 +61,8 @@
        ::yres/body data}
 
       (binding [l/*context* (request->context request)]
-        (l/err :hint "restriction error" :data data)
+        (l/err :hint "restriction error"
+               :cause err)
         {::yres/status 400
          ::yres/body data}))))
 
@@ -102,7 +102,7 @@
       (= code :invalid-image)
       (binding [l/*context* (request->context request)]
         (let [cause (or parent-cause err)]
-          (l/warn :hint "unexpected error on processing image" :cause cause)
+          (l/warn :hint "image process error" :cause cause)
           {::yres/status 400 ::yres/body data}))
 
       :else
@@ -177,7 +177,7 @@
   (let [state (.getSQLState ^java.sql.SQLException error)
         cause (or parent-cause error)]
     (binding [l/*context* (request->context request)]
-      (l/error :hint "PSQL error"
+      (l/error :hint "postgresql error"
                :cause cause)
       (cond
         (= state "57014")

backend/src/app/http/session.clj

@@ -337,16 +337,17 @@
        or (updated_at is null and
            created_at < now() - ?::interval)")
 
-(defmethod ig/init-key ::tasks/gc
-  [_ {:keys [::db/pool ::tasks/max-age] :as cfg}]
-  (l/debug :hint "initializing session gc task" :max-age max-age)
-  (fn [_]
-    (db/with-atomic [conn pool]
-      (let [interval (db/interval max-age)
-            result   (db/exec-one! conn [sql:delete-expired interval interval])
-            result   (:next.jdbc/update-count result)]
-        (l/debug :task "gc"
-                 :hint "clean http sessions"
-                 :deleted result)
-        result))))
+(defn- collect-expired-tasks
+  [{:keys [::db/conn ::tasks/max-age]}]
+  (let [interval (db/interval max-age)
+        result   (db/exec-one! conn [sql:delete-expired interval interval])
+        result   (:next.jdbc/update-count result)]
+    (l/debug :task "gc"
+             :hint "clean http sessions"
+             :deleted result)
+    result))
 
+(defmethod ig/init-key ::tasks/gc
+  [_ {:keys [::tasks/max-age] :as cfg}]
+  (l/debug :hint "initializing session gc task" :max-age max-age)
+  (fn [_] (db/tx-run! cfg collect-expired-tasks)))

backend/src/app/http/sse.clj

@@ -9,7 +9,6 @@
   (:refer-clojure :exclude [tap])
   (:require
    [app.common.data :as d]
-   [app.common.exceptions :as ex]
    [app.common.logging :as l]
    [app.common.transit :as t]
    [app.http.errors :as errors]
@@ -54,18 +53,20 @@
      ::yres/status 200
      ::yres/body (yres/stream-body
                   (fn [_ output]
-                    (binding [events/*channel* (sp/chan :buf buf :xf (keep encode))]
-                      (let [listener (events/start-listener
-                                      (partial write! output)
-                                      (partial pu/close! output))]
-                        (try
+                    (let [channel  (sp/chan :buf buf :xf (keep encode))
+                          listener (events/start-listener
+                                    channel
+                                    (partial write! output)
+                                    (partial pu/close! output))]
+                      (try
+                        (binding [events/*channel* channel]
                           (let [result (handler)]
-                            (events/tap :end result))
-                          (catch Throwable cause
-                            (events/tap :error (errors/handle' cause request))
-                            (when-not (ex/instance? java.io.EOFException cause)
-                              (binding [l/*context* (errors/request->context request)]
-                                (l/err :hint "unexpected error on processing sse response" :cause cause))))
-                          (finally
-                            (sp/close! events/*channel*)
-                            (px/await! listener)))))))}))
+                            (events/tap :end result)))
+
+                        (catch Throwable cause
+                          (let [result (errors/handle' cause request)]
+                            (events/tap channel :error result)))
+
+                        (finally
+                          (sp/close! channel)
+                          (px/await! listener))))))}))

backend/src/app/http/websocket.clj

@@ -273,7 +273,7 @@
 
 (defn- http-handler
   [cfg {:keys [params ::session/profile-id] :as request}]
-  (let [session-id (some-> params :session-id sm/parse-uuid)]
+  (let [session-id (some-> params :session-id uuid/parse*)]
     (when-not (uuid? session-id)
       (ex/raise :type :validation
                 :code :missing-session-id

backend/src/app/loggers/audit.clj

@@ -108,6 +108,7 @@
    [::ip-addr {:optional true} ::sm/text]
    [::props {:optional true} [:map-of :keyword :any]]
    [::context {:optional true} [:map-of :keyword :any]]
+   [::tracked-at {:optional true} ::sm/inst]
    [::webhooks/event? {:optional true} ::sm/boolean]
    [::webhooks/batch-timeout {:optional true} ::dt/duration]
    [::webhooks/batch-key {:optional true}
@@ -118,12 +119,12 @@
 
 (defn prepare-event
   [cfg mdata params result]
-  (let [resultm    (meta result)
-        request    (-> params meta ::http/request)
-        profile-id (or (::profile-id resultm)
-                       (:profile-id result)
-                       (::rpc/profile-id params)
-                       uuid/zero)
+  (let [resultm      (meta result)
+        request      (-> params meta ::http/request)
+        profile-id   (or (::profile-id resultm)
+                         (:profile-id result)
+                         (::rpc/profile-id params)
+                         uuid/zero)
 
         session-id   (get params ::rpc/external-session-id)
         event-origin (get params ::rpc/external-event-origin)
@@ -135,14 +136,14 @@
 
                          (clean-props))
 
-        token-id  (::actoken/id request)
-        context   (-> (::context resultm)
-                      (assoc :external-session-id session-id)
-                      (assoc :external-event-origin event-origin)
-                      (assoc :access-token-id (some-> token-id str))
-                      (d/without-nils))
+        token-id     (::actoken/id request)
+        context      (-> (::context resultm)
+                         (assoc :external-session-id session-id)
+                         (assoc :external-event-origin event-origin)
+                         (assoc :access-token-id (some-> token-id str))
+                         (d/without-nils))
 
-        ip-addr   (inet/parse-request request)]
+        ip-addr      (inet/parse-request request)]
 
     {::type (or (::type resultm)
                 (::rpc/type cfg))

backend/src/app/loggers/database.clj

@@ -53,11 +53,16 @@
                    (assoc :logger/name logger)
                    (assoc :logger/level level)
                    (dissoc :request/params :value :params :data))]
+
       (merge
        {:context (-> (into (sorted-map) ctx)
                      (pp/pprint-str :length 50))
         :props   (pp/pprint-str props :length 50)
-        :hint    (or (ex-message cause) @message)
+        :hint    (or (when-let [message (ex-message cause)]
+                       (if-let [props-hint (:hint props)]
+                         (str props-hint ": " message)
+                         message))
+                     @message)
         :trace   (or (::trace record)
                      (some-> cause (ex/format-throwable :data? false :explain? false :header? false :summary? false)))}
 

backend/src/app/loggers/webhooks.clj

@@ -15,6 +15,7 @@
    [app.config :as cf]
    [app.db :as db]
    [app.http.client :as http]
+   [app.loggers.audit :as audit]
    [app.util.time :as dt]
    [app.worker :as wrk]
    [clojure.data.json :as json]
@@ -67,18 +68,27 @@
 (defmethod ig/init-key ::process-event-handler
   [_ cfg]
   (fn [{:keys [props] :as task}]
-    (l/dbg :hint "process webhook event" :name (:name props))
 
-    (when-let [items (lookup-webhooks cfg props)]
-      (l/trc :hint "webhooks found for event" :total (count items))
-      (db/tx-run! cfg (fn [cfg]
-                        (doseq [item items]
-                          (wrk/submit! (-> cfg
-                                           (assoc ::wrk/task :run-webhook)
-                                           (assoc ::wrk/queue :webhooks)
-                                           (assoc ::wrk/max-retries 3)
-                                           (assoc ::wrk/params {:event props
-                                                                :config item})))))))))
+    (let [items (lookup-webhooks cfg props)
+          event {::audit/profile-id (:profile-id props)
+                 ::audit/name "webhook"
+                 ::audit/type "trigger"
+                 ::audit/props {:name (get props :name)
+                                :event-id (get props :id)
+                                :total-affected (count items)}}]
+
+      (audit/insert! cfg event)
+
+      (when items
+        (l/trc :hint "webhooks found for event" :total (count items))
+        (db/tx-run! cfg (fn [cfg]
+                          (doseq [item items]
+                            (wrk/submit! (-> cfg
+                                             (assoc ::wrk/task :run-webhook)
+                                             (assoc ::wrk/queue :webhooks)
+                                             (assoc ::wrk/max-retries 3)
+                                             (assoc ::wrk/params {:event props
+                                                                  :config item}))))))))))
 ;; --- RUN
 
 (declare interpret-exception)

backend/src/app/media.clj

@@ -8,12 +8,11 @@
   "Media & Font postprocessing."
   (:require
    [app.common.data :as d]
+   [app.common.data.macros :as dm]
    [app.common.exceptions :as ex]
    [app.common.media :as cm]
    [app.common.schema :as sm]
    [app.common.schema.openapi :as-alias oapi]
-   [app.common.spec :as us]
-   [app.common.svg :as csvg]
    [app.config :as cf]
    [app.db :as-alias db]
    [app.storage :as-alias sto]
@@ -22,39 +21,38 @@
    [buddy.core.bytes :as bb]
    [buddy.core.codecs :as bc]
    [clojure.java.shell :as sh]
-   [clojure.spec.alpha :as s]
+   [clojure.xml :as xml]
    [cuerdas.core :as str]
    [datoteka.fs :as fs]
    [datoteka.io :as io])
   (:import
+   clojure.lang.XMLHandler
+   java.io.InputStream
+   javax.xml.XMLConstants
+   javax.xml.parsers.SAXParserFactory
+   org.apache.commons.io.IOUtils
    org.im4java.core.ConvertCmd
    org.im4java.core.IMOperation
    org.im4java.core.Info))
 
-(s/def ::path fs/path?)
-(s/def ::filename string?)
-(s/def ::size integer?)
-(s/def ::headers (s/map-of string? string?))
-(s/def ::mtype string?)
+(def schema:upload
+  (sm/register!
+   ^{::sm/type ::upload}
+   [:map {:title "Upload"}
+    [:filename :string]
+    [:size ::sm/int]
+    [:path ::fs/path]
+    [:mtype {:optional true} :string]
+    [:headers {:optional true}
+     [:map-of :string :string]]]))
 
-(s/def ::upload
-  (s/keys :req-un [::filename ::size ::path]
-          :opt-un [::mtype ::headers]))
+(def ^:private schema:input
+  [:map {:title "Input"}
+   [:path ::fs/path]
+   [:mtype {:optional true} ::sm/text]])
 
-;; A subset of fields from the ::upload spec
-(s/def ::input
-  (s/keys :req-un [::path]
-          :opt-un [::mtype]))
-
-(sm/register!
- ^{::sm/type ::upload}
- [:map {:title "Upload"}
-  [:filename :string]
-  [:size ::sm/int]
-  [:path ::fs/path]
-  [:mtype {:optional true} :string]
-  [:headers {:optional true}
-   [:map-of :string :string]]])
+(def ^:private check-input
+  (sm/check-fn schema:input))
 
 (defn validate-media-type!
   ([upload] (validate-media-type! upload cm/valid-image-types))
@@ -97,17 +95,44 @@
     (catch Throwable e
       (process-error e))))
 
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; SVG PARSING
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn- secure-parser-factory
+  [^InputStream input ^XMLHandler handler]
+  (.. (doto (SAXParserFactory/newInstance)
+        (.setFeature XMLConstants/FEATURE_SECURE_PROCESSING true)
+        (.setFeature "http://apache.org/xml/features/disallow-doctype-decl" true))
+      (newSAXParser)
+      (parse input handler)))
+
+(defn- strip-doctype
+  [data]
+  (cond-> data
+    (str/includes? data "<!DOCTYPE")
+    (str/replace #"<\!DOCTYPE[^>]*>" "")))
+
+(defn- parse-svg
+  [text]
+  (let [text (strip-doctype text)]
+    (dm/with-open [istream (IOUtils/toInputStream text "UTF-8")]
+      (xml/parse istream secure-parser-factory))))
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; IMAGE THUMBNAILS
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
-(s/def ::width integer?)
-(s/def ::height integer?)
-(s/def ::format #{:jpeg :webp :png})
-(s/def ::quality #(< 0 % 101))
+(def ^:private schema:thumbnail-params
+  [:map {:title "ThumbnailParams"}
+   [:input schema:input]
+   [:format [:enum :jpeg :webp :png]]
+   [:quality [:int {:min 1 :max 100}]]
+   [:width :int]
+   [:height :int]])
 
-(s/def ::thumbnail-params
-  (s/keys :req-un [::input ::format ::width ::height]))
+(def ^:private check-thumbnail-params
+  (sm/check-fn schema:thumbnail-params))
 
 ;; Related info on how thumbnails generation
 ;;  http://www.imagemagick.org/Usage/thumbnails/
@@ -129,30 +154,38 @@
            :data   tmp)))
 
 (defmethod process :generic-thumbnail
-  [{:keys [quality width height] :as params}]
-  (us/assert ::thumbnail-params params)
-  (let [op (doto (IMOperation.)
-             (.addImage)
-             (.autoOrient)
-             (.strip)
-             (.thumbnail ^Integer (int width) ^Integer (int height) ">")
-             (.quality (double quality))
-             (.addImage))]
-    (generic-process (assoc params :operation op))))
+  [params]
+  (let [{:keys [quality width height] :as params}
+        (check-thumbnail-params params)
+
+        operation
+        (doto (IMOperation.)
+          (.addImage)
+          (.autoOrient)
+          (.strip)
+          (.thumbnail ^Integer (int width) ^Integer (int height) ">")
+          (.quality (double quality))
+          (.addImage))]
+
+    (generic-process (assoc params :operation operation))))
 
 (defmethod process :profile-thumbnail
-  [{:keys [quality width height] :as params}]
-  (us/assert ::thumbnail-params params)
-  (let [op (doto (IMOperation.)
-             (.addImage)
-             (.autoOrient)
-             (.strip)
-             (.thumbnail ^Integer (int width) ^Integer (int height) "^")
-             (.gravity "center")
-             (.extent (int width) (int height))
-             (.quality (double quality))
-             (.addImage))]
-    (generic-process (assoc params :operation op))))
+  [params]
+  (let [{:keys [quality width height] :as params}
+        (check-thumbnail-params params)
+
+        operation
+        (doto (IMOperation.)
+          (.addImage)
+          (.autoOrient)
+          (.strip)
+          (.thumbnail ^Integer (int width) ^Integer (int height) "^")
+          (.gravity "center")
+          (.extent (int width) (int height))
+          (.quality (double quality))
+          (.addImage))]
+
+    (generic-process (assoc params :operation operation))))
 
 (defn get-basic-info-from-svg
   [{:keys [tag attrs] :as data}]
@@ -184,10 +217,9 @@
 
 (defmethod process :info
   [{:keys [input] :as params}]
-  (us/assert ::input input)
-  (let [{:keys [path mtype]} input]
+  (let [{:keys [path mtype] :as input} (check-input input)]
     (if (= mtype "image/svg+xml")
-      (let [info (some-> path slurp csvg/parse get-basic-info-from-svg)]
+      (let [info (some-> path slurp parse-svg get-basic-info-from-svg)]
         (when-not info
           (ex/raise :type :validation
                     :code :invalid-svg-file

backend/src/app/rpc/commands/access_token.clj

@@ -43,13 +43,8 @@
     (decode-row token)))
 
 (defn repl:create-access-token
-  [{:keys [::db/pool] :as system} profile-id name expiration]
-  (db/with-atomic [conn pool]
-    (let [props (:app.setup/props system)]
-      (create-access-token {::db/conn conn ::setup/props props}
-                           profile-id
-                           name
-                           expiration))))
+  [cfg profile-id name expiration]
+  (db/tx-run! cfg create-access-token profile-id name expiration))
 
 (def ^:private schema:create-access-token
   [:map {:title "create-access-token"}

backend/src/app/rpc/commands/audit.clj

@@ -92,9 +92,9 @@
      [:string {:max 250}]
      [::sm/one-of {:format "string"} valid-event-types]]]
    [:props
-    [:map-of :keyword :any]]
+    [:map-of :keyword ::sm/any]]
    [:context {:optional true}
-    [:map-of :keyword :any]]])
+    [:map-of :keyword ::sm/any]]])
 
 (def schema:push-audit-events
   [:map {:title "push-audit-events"}

backend/src/app/rpc/commands/auth.clj

@@ -55,7 +55,7 @@
                 (contains? cf/flags :login-with-password))
     (ex/raise :type :restriction
               :code :login-disabled
-              :hint "login is disabled in this instance"))
+              :hint "login is disabled"))
 
   (letfn [(check-password [cfg profile password]
             (if (= (:password profile) "!")
@@ -79,7 +79,8 @@
                         :code :wrong-credentials))
             (when (:is-blocked profile)
               (ex/raise :type :restriction
-                        :code :profile-blocked))
+                        :code :profile-blocked
+                        :hint "profile is marked as blocked"))
             (when-not (check-password cfg profile password)
               (ex/raise :type :validation
                         :code :wrong-credentials))
@@ -149,7 +150,7 @@
 ;; ---- COMMAND: Recover Profile
 
 (defn recover-profile
-  [{:keys [::db/pool] :as cfg} {:keys [token password]}]
+  [{:keys [::db/conn] :as cfg} {:keys [token password]}]
   (letfn [(validate-token [token]
             (let [tdata (tokens/verify (::setup/props cfg) {:token token :iss :password-recovery})]
               (:profile-id tdata)))
@@ -159,10 +160,10 @@
               (db/update! conn :profile {:password pwd :is-active true} {:id profile-id})
               nil))]
 
-    (db/with-atomic [conn pool]
-      (->> (validate-token token)
-           (update-password conn))
-      nil)))
+    (->> (validate-token token)
+         (update-password conn))
+
+    nil))
 
 (def schema:recover-profile
   [:map {:title "recover-profile"}
@@ -173,7 +174,8 @@
   {::rpc/auth false
    ::doc/added "1.15"
    ::sm/params schema:recover-profile
-   ::climit/id :auth/global}
+   ::climit/id :auth/global
+   ::db/transaction true}
   [cfg params]
   (recover-profile cfg params))
 
@@ -182,11 +184,11 @@
 (defn- validate-register-attempt!
   [cfg params]
 
-  (when (or
-         (not (contains? cf/flags :registration))
-         (not (contains? cf/flags :login-with-password)))
+  (when (or (not (contains? cf/flags :registration))
+            (not (contains? cf/flags :login-with-password)))
     (ex/raise :type :restriction
-              :code :registration-disabled))
+              :code :registration-disabled
+              :hint "registration disabled"))
 
   (when (contains? params :invitation-token)
     (let [invitation (tokens/verify (::setup/props cfg)
@@ -200,12 +202,14 @@
   (when (and (email.blacklist/enabled? cfg)
              (email.blacklist/contains? cfg (:email params)))
     (ex/raise :type :restriction
-              :code :email-domain-is-not-allowed))
+              :code :email-domain-is-not-allowed
+              :hint "email domain in blacklist"))
 
   (when (and (email.whitelist/enabled? cfg)
              (not (email.whitelist/contains? cfg (:email params))))
     (ex/raise :type :restriction
-              :code :email-domain-is-not-allowed))
+              :code :email-domain-is-not-allowed
+              :hint "email domain not in whitelist"))
 
   ;; Perform a basic validation of email & password
   (when (= (str/lower (:email params))
@@ -218,16 +222,16 @@
     (ex/raise :type :restriction
               :code :email-has-permanent-bounces
               :email (:email params)
-              :hint "looks like the email has bounce reports"))
+              :hint "email has bounce reports"))
 
   (when (eml/has-complaint-reports? cfg (:email params))
     (ex/raise :type :restriction
               :code :email-has-complaints
               :email (:email params)
-              :hint "looks like the email has complaint reports")))
+              :hint "email has complaint reports")))
 
 (defn prepare-register
-  [{:keys [::db/pool] :as cfg} {:keys [email] :as params}]
+  [{:keys [::db/pool] :as cfg} {:keys [email accept-newsletter-updates] :as params}]
 
   (validate-register-attempt! cfg params)
 
@@ -239,7 +243,8 @@
                  :backend "penpot"
                  :iss :prepared-register
                  :profile-id (:id profile)
-                 :exp (dt/in-future {:days 7})}
+                 :exp (dt/in-future {:days 7})
+                 :props {:newsletter-updates (or accept-newsletter-updates false)}}
 
         params (d/without-nils params)
         token  (tokens/generate (::setup/props cfg) params)]

backend/src/app/rpc/commands/binfile.clj

@@ -38,7 +38,6 @@
 (def ^:private
   schema:export-binfile
   [:map {:title "export-binfile"}
-   [:name [:string {:max 250}]]
    [:file-id ::sm/uuid]
    [:version {:optional true} ::sm/int]
    [:include-libraries ::sm/boolean]
@@ -78,7 +77,7 @@
   "Export a penpot file in a binary format."
   {::doc/added "1.15"
    ::webhooks/event? true
-   ::sm/result schema:export-binfile}
+   ::sm/params schema:export-binfile}
   [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id version file-id] :as params}]
   (files/check-read-permissions! pool profile-id file-id)
   (fn [_]
@@ -116,7 +115,8 @@
 
     (db/update! pool :project
                 {:modified-at (dt/now)}
-                {:id project-id})
+                {:id project-id}
+                {::db/return-keys false})
 
     result))
 

backend/src/app/rpc/commands/comments.clj

@@ -797,3 +797,18 @@
                 {:id id}
                 {::db/return-keys false})
     nil))
+
+(def ^:private
+  schema:mark-all-threads-as-read
+  [:map {:title "mark-all-threads-as-read"}
+   [:threads [:vector ::sm/uuid]]])
+
+(sv/defmethod ::mark-all-threads-as-read
+  {::doc/added "1.15"
+   ::sm/params schema:mark-all-threads-as-read}
+  [cfg {:keys [::rpc/profile-id threads] :as params}]
+  (db/tx-run!
+   cfg
+   (fn [{:keys [::db/conn]}]
+     (doseq [thread-id threads]
+       (upsert-comment-thread-status! conn profile-id thread-id)))))

backend/src/app/rpc/commands/demo.clj

@@ -27,7 +27,7 @@
   {::rpc/auth false
    ::doc/added "1.15"
    ::doc/changes ["1.15" "This method is migrated from mutations to commands."]}
-  [{:keys [::db/pool] :as cfg} _]
+  [cfg _]
 
   (when-not (contains? cf/flags :demo-users)
     (ex/raise :type :validation
@@ -49,9 +49,11 @@
                   :password (profile/derive-password cfg password)
                   :props {}}]
 
-    (db/with-atomic [conn pool]
-      (let [profile (->> (auth/create-profile! conn params)
-                         (auth/create-profile-rels! conn))]
-        (with-meta {:email email
-                    :password password}
-          {::audit/profile-id (:id profile)})))))
+
+    (let [profile (db/tx-run! cfg (fn [{:keys [::db/conn]}]
+                                    (->> (auth/create-profile! conn params)
+                                         (auth/create-profile-rels! conn))))]
+      (with-meta {:email email
+                  :password password}
+        {::audit/profile-id (:id profile)}))))
+

backend/src/app/rpc/commands/files.clj

@@ -189,7 +189,7 @@
    [:is-shared ::sm/boolean]
    [:project-id ::sm/uuid]
    [:created-at ::dt/instant]
-   [:data {:optional true} :any]])
+   [:data {:optional true} ::sm/any]])
 
 (def schema:permissions-mixin
   [:map {:title "PermissionsMixin"}
@@ -208,7 +208,7 @@
    [:project-id {:optional true} ::sm/uuid]])
 
 (defn- migrate-file
-  [{:keys [::db/conn] :as cfg} {:keys [id] :as file}]
+  [{:keys [::db/conn] :as cfg} {:keys [id] :as file} {:keys [read-only?]}]
   (binding [pmap/*load-fn* (partial feat.fdata/load-pointer cfg id)
             pmap/*tracked* (pmap/create-tracked)]
     (let [;; For avoid unnecesary overhead of creating multiple pointers and
@@ -219,43 +219,45 @@
           file (-> file
                    (update :data feat.fdata/process-pointers deref)
                    (update :data feat.fdata/process-objects (partial into {}))
-                   (fmg/migrate-file))
+                   (fmg/migrate-file))]
 
-          ;; When file is migrated, we break the rule of no perform
-          ;; mutations on get operations and update the file with all
-          ;; migrations applied
-          ;;
-          ;; WARN: he following code will not work on read-only mode,
-          ;; it is a known issue; we keep is not implemented until we
-          ;; really need this.
-          file (if (contains? (:features file) "fdata/objects-map")
-                 (feat.fdata/enable-objects-map file)
-                 file)
-          file (if (contains? (:features file) "fdata/pointer-map")
-                 (feat.fdata/enable-pointer-map file)
-                 file)]
+      (if (or read-only? (db/read-only? conn))
+        file
+        (let [;; When file is migrated, we break the rule of no perform
+              ;; mutations on get operations and update the file with all
+              ;; migrations applied
+              file (if (contains? (:features file) "fdata/objects-map")
+                     (feat.fdata/enable-objects-map file)
+                     file)
+              file (if (contains? (:features file) "fdata/pointer-map")
+                     (feat.fdata/enable-pointer-map file)
+                     file)]
 
-      (db/update! conn :file
-                  {:data (blob/encode (:data file))
-                   :version (:version file)
-                   :features (db/create-array conn "text" (:features file))}
-                  {:id id})
+          (db/update! conn :file
+                      {:data (blob/encode (:data file))
+                       :version (:version file)
+                       :features (db/create-array conn "text" (:features file))}
+                      {:id id}
+                      {::db/return-keys false})
 
-      (when (contains? (:features file) "fdata/pointer-map")
-        (feat.fdata/persist-pointers! cfg id))
+          (when (contains? (:features file) "fdata/pointer-map")
+            (feat.fdata/persist-pointers! cfg id))
 
-      (feat.fmigr/upsert-migrations! conn file)
-      (feat.fmigr/resolve-applied-migrations cfg file))))
+          (feat.fmigr/upsert-migrations! conn file)
+          (feat.fmigr/resolve-applied-migrations cfg file))))))
 
 (defn get-file
   [{:keys [::db/conn ::wrk/executor] :as cfg} id
    & {:keys [project-id
              migrate?
              include-deleted?
-             lock-for-update?]
+             lock-for-update?
+             preload-pointers?]
       :or {include-deleted? false
            lock-for-update? false
-           migrate? true}}]
+           migrate? true
+           preload-pointers? false}
+      :as options}]
 
   (assert (db/connection? conn) "expected cfg with valid connection")
 
@@ -273,10 +275,16 @@
         ;; because it has heavy and synchronous operations for
         ;; decoding file body that are not very friendly with virtual
         ;; threads.
-        file   (px/invoke! executor #(decode-row file))]
+        file   (px/invoke! executor #(decode-row file))
+
+        file   (if (and migrate? (fmg/need-migration? file))
+                 (migrate-file cfg file options)
+                 file)]
+
+    (if preload-pointers?
+      (binding [pmap/*load-fn* (partial feat.fdata/load-pointer cfg id)]
+        (update file :data feat.fdata/process-pointers deref))
 
-    (if (and migrate? (fmg/need-migration? file))
-      (migrate-file cfg file)
       file)))
 
 (defn get-minimal-file
@@ -292,7 +300,7 @@
 
 (defn get-file-etag
   [{:keys [::rpc/profile-id]} {:keys [modified-at revn vern permissions]}]
-  (str profile-id "/" revn "/" vern "/"
+  (str profile-id "/" revn "/" vern "/" (hash fmg/available-migrations) "/"
        (dt/format-instant modified-at :iso)
        "/"
        (uri/map->query-string permissions)))
@@ -323,11 +331,12 @@
 
           file (-> (get-file cfg id :project-id project-id)
                    (assoc :permissions perms)
+                   (assoc :team-id (:id team))
                    (check-version!))]
 
       (-> (cfeat/get-team-enabled-features cf/flags team)
           (cfeat/check-client-features! (:features params))
-          (cfeat/check-file-features! (:features file) (:features params)))
+          (cfeat/check-file-features! (:features file)))
 
       ;; This operation is needed for backward comapatibility with frontends that
       ;; does not support pointer-map resolution mechanism; this just resolves the
@@ -473,7 +482,7 @@
   (update page :objects update-vals #(dissoc % :thumbnail)))
 
 (defn get-page
-  [{:keys [::db/conn] :as cfg} {:keys [profile-id file-id page-id object-id] :as params}]
+  [{:keys [::db/conn] :as cfg} {:keys [profile-id file-id page-id object-id share-id] :as params}]
 
   (when (and (uuid? object-id)
              (not (uuid? page-id)))
@@ -481,22 +490,30 @@
               :code :params-validation
               :hint "page-id is required when object-id is provided"))
 
-  (let [team (teams/get-team conn
-                             :profile-id profile-id
-                             :file-id file-id)
+  (let [perms (get-permissions conn profile-id file-id share-id)
 
-        file (get-file cfg file-id)
+        file  (get-file cfg file-id :read-only? true)
 
-        _    (-> (cfeat/get-team-enabled-features cf/flags team)
-                 (cfeat/check-client-features! (:features params))
-                 (cfeat/check-file-features! (:features file) (:features params)))
+        proj  (db/get conn :project {:id (:project-id file)})
 
-        page (binding [pmap/*load-fn* (partial feat.fdata/load-pointer cfg file-id)]
-               (let [page-id (or page-id (-> file :data :pages first))
-                     page    (dm/get-in file [:data :pages-index page-id])]
-                 (if (pmap/pointer-map? page)
-                   (deref page)
-                   page)))]
+        team  (-> (db/get conn :team {:id (:team-id proj)})
+                  (teams/decode-row))
+
+        _     (-> (cfeat/get-team-enabled-features cf/flags team)
+                  (cfeat/check-client-features! (:features params))
+                  (cfeat/check-file-features! (:features file)))
+
+        page  (binding [pmap/*load-fn* (partial feat.fdata/load-pointer cfg file-id)]
+                (let [page-id (or page-id (-> file :data :pages first))
+                      page    (dm/get-in file [:data :pages-index page-id])]
+                  (if (pmap/pointer-map? page)
+                    (deref page)
+                    page)))]
+
+    (when-not perms
+      (ex/raise :type :not-found
+                :code :object-not-found
+                :hint "object not found"))
 
     (cond-> (prune-thumbnails page)
       (some? object-id)
@@ -613,6 +630,7 @@
    SELECT l.id,
           l.features,
           l.project_id,
+          p.team_id,
           l.created_at,
           l.modified_at,
           l.deleted_at,
@@ -622,6 +640,7 @@
           l.synced_at,
           l.is_shared
      FROM libs AS l
+    INNER JOIN project AS p ON (p.id = l.project_id)
     WHERE l.deleted_at IS NULL OR l.deleted_at > now();")
 
 (defn get-file-libraries
@@ -730,11 +749,13 @@
                              :project-id project-id
                              :file-id id)
 
-        file (get-file cfg id :project-id project-id)]
+        file (get-file cfg id
+                       :project-id project-id
+                       :read-only? true)]
 
     (-> (cfeat/get-team-enabled-features cf/flags team)
         (cfeat/check-client-features! (:features params))
-        (cfeat/check-file-features! (:features file) (:features params)))
+        (cfeat/check-file-features! (:features file)))
 
     (binding [pmap/*load-fn* (partial feat.fdata/load-pointer cfg id)]
       {:name             (:name file)
@@ -803,17 +824,17 @@
     [:id ::sm/uuid]
     [:name [:string {:max 250}]]
     [:created-at ::dt/instant]
-    [:modified-at ::dt/instant]]}
+    [:modified-at ::dt/instant]]
 
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id] :as params}]
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id id)
-    (let [file (rename-file conn params)]
-      (rph/with-meta
-        (select-keys file [:id :name :created-at :modified-at])
-        {::audit/props {:project-id (:project-id file)
-                        :created-at (:created-at file)
-                        :modified-at (:modified-at file)}}))))
+   ::db/transaction true}
+  [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id id] :as params}]
+  (check-edition-permissions! conn profile-id id)
+  (let [file (rename-file conn params)]
+    (rph/with-meta
+      (select-keys file [:id :name :created-at :modified-at])
+      {::audit/props {:project-id (:project-id file)
+                      :created-at (:created-at file)
+                      :modified-at (:modified-at file)}})))
 
 ;; --- MUTATION COMMAND: set-file-shared
 
@@ -1005,15 +1026,17 @@
   {::doc/added "1.17"
    ::webhooks/event? true
    ::sm/params schema:link-file-to-library}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id library-id] :as params}]
+  [cfg {:keys [::rpc/profile-id file-id library-id] :as params}]
   (when (= file-id library-id)
     (ex/raise :type :validation
               :code :invalid-library
               :hint "A file cannot be linked to itself"))
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id file-id)
-    (check-edition-permissions! conn profile-id library-id)
-    (link-file-to-library conn params)))
+
+  (db/tx-run! cfg
+              (fn [{:keys [::db/conn]}]
+                (check-edition-permissions! conn profile-id file-id)
+                (check-edition-permissions! conn profile-id library-id)
+                (link-file-to-library conn params))))
 
 ;; --- MUTATION COMMAND: unlink-file-from-library
 
@@ -1031,12 +1054,12 @@
 (sv/defmethod ::unlink-file-from-library
   {::doc/added "1.17"
    ::webhooks/event? true
-   ::sm/params schema:unlink-file-to-library}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id file-id)
-    (unlink-file-from-library conn params)
-    nil))
+   ::sm/params schema:unlink-file-to-library
+   ::db/transaction true}
+  [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
+  (check-edition-permissions! conn profile-id file-id)
+  (unlink-file-from-library conn params)
+  nil)
 
 ;; --- MUTATION COMMAND: update-sync
 
@@ -1056,12 +1079,11 @@
 (sv/defmethod ::update-file-library-sync-status
   "Update the synchronization status of a file->library link"
   {::doc/added "1.17"
-   ::sm/params schema:update-file-library-sync-status}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id file-id)
-    (update-sync conn params)))
-
+   ::sm/params schema:update-file-library-sync-status
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id file-id] :as params}]
+  (check-edition-permissions! conn profile-id file-id)
+  (update-sync conn params))
 
 ;; --- MUTATION COMMAND: ignore-sync
 
@@ -1082,9 +1104,9 @@
 (sv/defmethod ::ignore-file-library-sync-status
   "Ignore updates in linked files"
   {::doc/added "1.17"
-   ::sm/params schema:ignore-file-library-sync-status}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id file-id)
-    (->  (ignore-sync conn params)
-         (update :features db/decode-pgarray #{}))))
+   ::sm/params schema:ignore-file-library-sync-status
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id file-id] :as params}]
+  (check-edition-permissions! conn profile-id file-id)
+  (->  (ignore-sync conn params)
+       (update :features db/decode-pgarray #{})))

backend/src/app/rpc/commands/files_create.clj

@@ -55,8 +55,8 @@
                                :features features
                                :ignore-sync-until ignore-sync-until
                                :modified-at modified-at
-                               :deleted-at deleted-at
-                               :create-page create-page
+                               :deleted-at deleted-at}
+                              {:create-page create-page
                                :page-id page-id})
           file (-> (bfc/insert-file! cfg file)
                    (bfc/decode-row))]
@@ -91,9 +91,6 @@
                                  :project-id project-id)
         team-id  (:id team)
 
-        ;; When we create files, we only need to respect the team
-        ;; features, because some features can be enabled
-        ;; globally, but the team is still not migrated properly.
         features (-> (cfeat/get-team-enabled-features cf/flags team)
                      (cfeat/check-client-features! (:features params)))
 
@@ -107,25 +104,28 @@
 
         params   (-> params
                      (assoc :profile-id profile-id)
-                     (assoc :features (set/difference features cfeat/frontend-only-features)))]
+                     (assoc :features features))]
 
     (quotes/check! cfg {::quotes/id ::quotes/files-per-project
                         ::quotes/team-id team-id
                         ::quotes/profile-id profile-id
                         ::quotes/project-id project-id})
 
-    ;; FIXME: IMPORTANT: this code can have race
-    ;; conditions, because we have no locks for updating
-    ;; team so, creating two files concurrently can lead
-    ;; to lost team features updating
+    ;; FIXME: IMPORTANT: this code can have race conditions, because
+    ;; we have no locks for updating team so, creating two files
+    ;; concurrently can lead to lost team features updating
 
-    ;; When newly computed features does not match exactly with
-    ;; the features defined on team row, we update it.
-    (when (not= features (:features team))
-      (let [features (db/create-array conn "text" features)]
+    (when-let [features (-> features
+                            (set/difference (:features team))
+                            (set/difference cfeat/no-team-inheritable-features)
+                            (not-empty))]
+      (let [features (->> features
+                          (set/union (:features team))
+                          (db/create-array conn "text"))]
         (db/update! conn :team
                     {:features features}
-                    {:id team-id})))
+                    {:id (:id team)}
+                    {::db/return-keys false})))
 
     (-> (create-file cfg params)
         (vary-meta assoc ::audit/props {:team-id team-id}))))

backend/src/app/rpc/commands/files_share.clj

@@ -33,11 +33,11 @@
   pages of a file with specific permissions (who-comment and who-inspect)."
   {::doc/added "1.18"
    ::doc/module :files
-   ::sm/params schema:create-share-link}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
-  (db/with-atomic [conn pool]
-    (files/check-edition-permissions! conn profile-id file-id)
-    (create-share-link conn (assoc params :profile-id profile-id))))
+   ::sm/params schema:create-share-link
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id file-id] :as params}]
+  (files/check-edition-permissions! conn profile-id file-id)
+  (create-share-link conn (assoc params :profile-id profile-id)))
 
 (defn create-share-link
   [conn {:keys [profile-id file-id pages who-comment who-inspect]}]
@@ -61,10 +61,10 @@
 (sv/defmethod ::delete-share-link
   {::doc/added "1.18"
    ::doc/module ::files
-   ::sm/params schema:delete-share-link}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id] :as params}]
-  (db/with-atomic [conn pool]
-    (let [slink (db/get-by-id conn :share-link id)]
-      (files/check-edition-permissions! conn profile-id (:file-id slink))
-      (db/delete! conn :share-link {:id id})
-      nil)))
+   ::sm/params schema:delete-share-link
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id id] :as params}]
+  (let [slink (db/get-by-id conn :share-link id)]
+    (files/check-edition-permissions! conn profile-id (:file-id slink))
+    (db/delete! conn :share-link {:id id})
+    nil))

backend/src/app/rpc/commands/files_temp.clj

@@ -14,7 +14,6 @@
    [app.config :as cf]
    [app.db :as db]
    [app.db.sql :as sql]
-   [app.features.components-v2 :as feat.compv2]
    [app.features.fdata :as fdata]
    [app.loggers.audit :as audit]
    [app.rpc :as-alias rpc]
@@ -110,7 +109,7 @@
 ;; --- MUTATION COMMAND: persist-temp-file
 
 (defn persist-temp-file
-  [{:keys [::db/conn] :as cfg} {:keys [id ::rpc/profile-id] :as params}]
+  [{:keys [::db/conn] :as cfg} {:keys [id] :as params}]
   (let [file (files/get-file cfg id
                              :migrate? false
                              :lock-for-update? true)]
@@ -119,7 +118,6 @@
       (ex/raise :type :validation
                 :code :cant-persist-already-persisted-file))
 
-
     (let [changes (->> (db/cursor conn
                                   (sql/select :file-change {:file-id id}
                                               {:order-by [[:revn :asc]]})
@@ -147,19 +145,6 @@
                    :revn 1
                    :data (blob/encode (:data file))}
                   {:id id})
-
-      (let [team          (teams/get-team conn :profile-id profile-id :project-id (:project-id file))
-            file-features (:features file)
-            team-features (cfeat/get-team-enabled-features cf/flags team)]
-        (when (and (contains? team-features "components/v2")
-                   (not (contains? file-features "components/v2")))
-          ;; Migrate components v2
-          (feat.compv2/migrate-file! cfg
-                                     (:id file)
-                                     :max-procs 2
-                                     :validate? true
-                                     :throw-on-validate? true)))
-
       nil)))
 
 (def ^:private schema:persist-temp-file

backend/src/app/rpc/commands/files_thumbnails.clj

@@ -10,7 +10,6 @@
    [app.common.data.macros :as dm]
    [app.common.features :as cfeat]
    [app.common.files.helpers :as cfh]
-   [app.common.files.migrations :as fmg]
    [app.common.geom.shapes :as gsh]
    [app.common.schema :as sm]
    [app.common.thumbnails :as thc]
@@ -18,7 +17,6 @@
    [app.config :as cf]
    [app.db :as db]
    [app.db.sql :as-alias sql]
-   [app.features.fdata :as feat.fdata]
    [app.loggers.audit :as-alias audit]
    [app.loggers.webhooks :as-alias webhooks]
    [app.media :as media]
@@ -180,8 +178,7 @@
 (def ^:private
   schema:get-file-data-for-thumbnail
   [:map {:title "get-file-data-for-thumbnail"}
-   [:file-id ::sm/uuid]
-   [:features {:optional true} ::cfeat/features]])
+   [:file-id ::sm/uuid]])
 
 (def ^:private
   schema:partial-file
@@ -201,18 +198,16 @@
   (db/run! cfg (fn [{:keys [::db/conn] :as cfg}]
                  (files/check-read-permissions! conn profile-id file-id)
 
-                 (let [team     (teams/get-team conn
-                                                :profile-id profile-id
-                                                :file-id file-id)
+                 (let [team (teams/get-team conn
+                                            :profile-id profile-id
+                                            :file-id file-id)
 
-                       file     (binding [pmap/*load-fn* (partial feat.fdata/load-pointer cfg file-id)]
-                                  (-> (files/get-file cfg file-id :migrate? false)
-                                      (update :data feat.fdata/process-pointers deref)
-                                      (fmg/migrate-file)))]
+                       file (files/get-file cfg file-id
+                                            :preload-pointers? true
+                                            :read-only? true)]
 
                    (-> (cfeat/get-team-enabled-features cf/flags team)
-                       (cfeat/check-client-features! (:features params))
-                       (cfeat/check-file-features! (:features file) (:features params)))
+                       (cfeat/check-file-features! (:features file)))
 
                    {:file-id file-id
                     :revn (:revn file)

backend/src/app/rpc/commands/files_update.clj

@@ -142,7 +142,7 @@
 
                           features (-> (cfeat/get-team-enabled-features cf/flags team)
                                        (cfeat/check-client-features! (:features params))
-                                       (cfeat/check-file-features! (:features file) (:features params)))
+                                       (cfeat/check-file-features! (:features file)))
 
                           changes  (if changes-with-metadata
                                      (->> changes-with-metadata (mapcat :changes) vec)
@@ -177,12 +177,19 @@
                                             :stored-revn (:revn file)}))
 
                       ;; When newly computed features does not match exactly with
-                      ;; the features defined on team row, we update it.
-                      (when (not= features (:features team))
-                        (let [features (db/create-array conn "text" features)]
+                      ;; the features defined on team row, we update it
+                      (when-let [features (-> features
+                                              (set/difference (:features team))
+                                              (set/difference cfeat/no-team-inheritable-features)
+                                              (not-empty))]
+                        (let [features (->> features
+                                            (set/union (:features team))
+                                            (db/create-array conn "text"))]
                           (db/update! conn :team
                                       {:features features}
-                                      {:id (:id team)})))
+                                      {:id (:id team)}
+                                      {::db/return-keys false})))
+
 
                       (mtx/run! metrics {:id :update-file-changes :inc (count changes)})
 

backend/src/app/rpc/commands/fonts.clj

@@ -80,9 +80,9 @@
 (def ^:private schema:create-font-variant
   [:map {:title "create-font-variant"}
    [:team-id ::sm/uuid]
-   [:data [:map-of :string :any]]
+   [:data [:map-of ::sm/text ::sm/any]]
    [:font-id ::sm/uuid]
-   [:font-family :string]
+   [:font-family ::sm/text]
    [:font-weight [::sm/one-of {:format "number"} valid-weight]]
    [:font-style [::sm/one-of {:format "string"} valid-style]]])
 

backend/src/app/rpc/commands/management.clj

@@ -396,45 +396,49 @@
 ;; --- COMMAND: Clone Template
 
 (defn clone-template
-  [cfg {:keys [project-id profile-id] :as params} template]
-  (db/tx-run! cfg (fn [{:keys [::db/conn ::wrk/executor] :as cfg}]
-                    ;; NOTE: the importation process performs some operations
-                    ;; that are not very friendly with virtual threads, and for
-                    ;; avoid unexpected blocking of other concurrent operations
-                    ;; we dispatch that operation to a dedicated executor.
-                    (let [template (tmp/tempfile-from template
-                                                      :prefix "penpot.template."
-                                                      :suffix ""
-                                                      :min-age "30m")
+  [{:keys [::db/pool ::wrk/executor] :as cfg} {:keys [project-id profile-id] :as params} template]
 
-                          format   (bfc/parse-file-format template)
-                          team     (teams/get-team conn
-                                                   :profile-id profile-id
-                                                   :project-id project-id)
-                          cfg      (-> cfg
-                                       (assoc ::bfc/project-id project-id)
-                                       (assoc ::bfc/profile-id profile-id)
-                                       (assoc ::bfc/input template)
-                                       (assoc ::bfc/features (cfeat/get-team-enabled-features cf/flags team)))
+  ;; NOTE: the importation process performs some operations
+  ;; that are not very friendly with virtual threads, and for
+  ;; avoid unexpected blocking of other concurrent operations
+  ;; we dispatch that operation to a dedicated executor.
+  (let [template (tmp/tempfile-from template
+                                    :prefix "penpot.template."
+                                    :suffix ""
+                                    :min-age "30m")
 
-                          result   (if (= format :binfile-v3)
-                                     (px/invoke! executor (partial bf.v3/import-files! cfg))
-                                     (px/invoke! executor (partial bf.v1/import-files! cfg)))]
+        format   (bfc/parse-file-format template)
+        team     (teams/get-team pool
+                                 :profile-id profile-id
+                                 :project-id project-id)
 
-                      (db/update! conn :project
-                                  {:modified-at (dt/now)}
-                                  {:id project-id})
+        cfg      (-> cfg
+                     (assoc ::bfc/project-id project-id)
+                     (assoc ::bfc/profile-id profile-id)
+                     (assoc ::bfc/input template)
+                     (assoc ::bfc/features (cfeat/get-team-enabled-features cf/flags team)))
 
-                      (let [props (audit/clean-props params)]
-                        (doseq [file-id result]
-                          (let [props (assoc props :id file-id)
-                                event (-> (audit/event-from-rpc-params params)
-                                          (assoc ::audit/profile-id profile-id)
-                                          (assoc ::audit/name "create-file")
-                                          (assoc ::audit/props props))]
-                            (audit/submit! cfg event))))
+        result   (if (= format :binfile-v3)
+                   (px/invoke! executor (partial bf.v3/import-files! cfg))
+                   (px/invoke! executor (partial bf.v1/import-files! cfg)))]
 
-                      result))))
+    (db/tx-run! cfg
+                (fn [{:keys [::db/conn] :as cfg}]
+                  (db/update! conn :project
+                              {:modified-at (dt/now)}
+                              {:id project-id}
+                              {::db/return-keys false})
+
+                  (let [props (audit/clean-props params)]
+                    (doseq [file-id result]
+                      (let [props (assoc props :id file-id)
+                            event (-> (audit/event-from-rpc-params params)
+                                      (assoc ::audit/profile-id profile-id)
+                                      (assoc ::audit/name "create-file")
+                                      (assoc ::audit/props props))]
+                        (audit/submit! cfg event))))))
+
+    result))
 
 (def ^:private
   schema:clone-template

backend/src/app/rpc/commands/media.clj

@@ -273,15 +273,14 @@
 
 (sv/defmethod ::clone-file-media-object
   {::doc/added "1.17"
-   ::sm/params schema:clone-file-media-object}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
-  (db/with-atomic [conn pool]
-    (files/check-edition-permissions! conn profile-id file-id)
-    (-> (assoc cfg :conn conn)
-        (clone-file-media-object params))))
+   ::sm/params schema:clone-file-media-object
+   ::db/transaction true}
+  [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
+  (files/check-edition-permissions! conn profile-id file-id)
+  (clone-file-media-object cfg params))
 
 (defn clone-file-media-object
-  [{:keys [conn]} {:keys [id file-id is-local]}]
+  [{:keys [::db/conn]} {:keys [id file-id is-local]}]
   (let [mobj (db/get-by-id conn :file-media-object id)]
     (db/insert! conn :file-media-object
                 {:id (uuid/next)

backend/src/app/rpc/commands/profile.clj

@@ -125,32 +125,32 @@
 (sv/defmethod ::update-profile
   {::doc/added "1.0"
    ::sm/params schema:update-profile
-   ::sm/result schema:profile}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id fullname lang theme] :as params}]
-  (db/with-atomic [conn pool]
-    ;; NOTE: we need to retrieve the profile independently if we use
-    ;; it or not for explicit locking and avoid concurrent updates of
-    ;; the same row/object.
-    (let [profile (-> (db/get-by-id conn :profile profile-id ::sql/for-update true)
-                      (decode-row))
+   ::sm/result schema:profile
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id fullname lang theme] :as params}]
+  ;; NOTE: we need to retrieve the profile independently if we use
+  ;; it or not for explicit locking and avoid concurrent updates of
+  ;; the same row/object.
+  (let [profile (-> (db/get-by-id conn :profile profile-id ::sql/for-update true)
+                    (decode-row))
 
-          ;; Update the profile map with direct params
-          profile (-> profile
-                      (assoc :fullname fullname)
-                      (assoc :lang lang)
-                      (assoc :theme theme))]
+        ;; Update the profile map with direct params
+        profile (-> profile
+                    (assoc :fullname fullname)
+                    (assoc :lang lang)
+                    (assoc :theme theme))]
 
-      (db/update! conn :profile
-                  {:fullname fullname
-                   :lang lang
-                   :theme theme
-                   :props (db/tjson (:props profile))}
-                  {:id profile-id})
+    (db/update! conn :profile
+                {:fullname fullname
+                 :lang lang
+                 :theme theme
+                 :props (db/tjson (:props profile))}
+                {:id profile-id})
 
-      (-> profile
-          (strip-private-attrs)
-          (d/without-nils)
-          (rph/with-meta {::audit/props (audit/profile->props profile)})))))
+    (-> profile
+        (strip-private-attrs)
+        (d/without-nils)
+        (rph/with-meta {::audit/props (audit/profile->props profile)}))))
 
 
 ;; --- MUTATION: Update Password
@@ -169,21 +169,20 @@
 (sv/defmethod ::update-profile-password
   {::doc/added "1.0"
    ::sm/params schema:update-profile-password
-   ::climit/id :auth/global}
+   ::climit/id :auth/global
+   ::db/transaction true}
   [cfg {:keys [::rpc/profile-id password] :as params}]
+  (let [profile    (validate-password! cfg (assoc params :profile-id profile-id))
+        session-id (::session/id params)]
 
-  (db/tx-run! cfg (fn [cfg]
-                    (let [profile    (validate-password! cfg (assoc params :profile-id profile-id))
-                          session-id (::session/id params)]
+    (when (= (:email profile) (str/lower (:password params)))
+      (ex/raise :type :validation
+                :code :email-as-password
+                :hint "you can't use your email as password"))
 
-                      (when (= (:email profile) (str/lower (:password params)))
-                        (ex/raise :type :validation
-                                  :code :email-as-password
-                                  :hint "you can't use your email as password"))
-
-                      (update-profile-password! cfg (assoc profile :password password))
-                      (invalidate-profile-session! cfg profile-id session-id)
-                      nil))))
+    (update-profile-password! cfg (assoc profile :password password))
+    (invalidate-profile-session! cfg profile-id session-id)
+    nil))
 
 (defn- invalidate-profile-session!
   "Removes all sessions except the current one."
@@ -441,37 +440,36 @@
 (declare ^:private get-owned-teams)
 
 (sv/defmethod ::delete-profile
-  {::doc/added "1.0"}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id] :as params}]
-  (db/with-atomic [conn pool]
-    (let [teams      (get-owned-teams conn profile-id)
-          deleted-at (dt/now)]
+  {::doc/added "1.0"
+   ::db/transaction true}
+  [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id] :as params}]
+  (let [teams      (get-owned-teams conn profile-id)
+        deleted-at (dt/now)]
 
-      ;; If we found owned teams with participants, we don't allow
-      ;; delete profile until the user properly transfer ownership or
-      ;; explicitly removes all participants from the team
-      (when (some pos? (map :participants teams))
-        (ex/raise :type :validation
-                  :code :owner-teams-with-people
-                  :hint "The user need to transfer ownership of owned teams."
-                  :context {:teams (mapv :id teams)}))
+    ;; If we found owned teams with participants, we don't allow
+    ;; delete profile until the user properly transfer ownership or
+    ;; explicitly removes all participants from the team
+    (when (some pos? (map :participants teams))
+      (ex/raise :type :validation
+                :code :owner-teams-with-people
+                :hint "The user need to transfer ownership of owned teams."
+                :context {:teams (mapv :id teams)}))
 
-      ;; Mark profile deleted immediatelly
-      (db/update! conn :profile
-                  {:deleted-at deleted-at}
-                  {:id profile-id})
+    ;; Mark profile deleted immediatelly
+    (db/update! conn :profile
+                {:deleted-at deleted-at}
+                {:id profile-id})
 
-      ;; Schedule cascade deletion to a worker
-      (wrk/submit! {::db/conn conn
-                    ::wrk/task :delete-object
-                    ::wrk/params {:object :profile
-                                  :deleted-at deleted-at
-                                  :id profile-id}})
+    ;; Schedule cascade deletion to a worker
+    (wrk/submit! {::db/conn conn
+                  ::wrk/task :delete-object
+                  ::wrk/params {:object :profile
+                                :deleted-at deleted-at
+                                :id profile-id}})
 
 
-      (-> (rph/wrap nil)
-          (rph/with-transform (session/delete-fn cfg))))))
-
+    (-> (rph/wrap nil)
+        (rph/with-transform (session/delete-fn cfg)))))
 
 ;; --- HELPERS
 

backend/src/app/rpc/commands/projects.clj

@@ -219,12 +219,12 @@
    ::sm/params schema:update-project-pin
    ::webhooks/batch-timeout (dt/duration "5s")
    ::webhooks/batch-key (webhooks/key-fn ::rpc/profile-id :id)
-   ::webhooks/event? true}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id team-id is-pinned] :as params}]
-  (db/with-atomic [conn pool]
-    (check-read-permissions! conn profile-id id)
-    (db/exec-one! conn [sql:update-project-pin team-id id profile-id is-pinned is-pinned])
-    nil))
+   ::webhooks/event? true
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id id team-id is-pinned] :as params}]
+  (check-read-permissions! conn profile-id id)
+  (db/exec-one! conn [sql:update-project-pin team-id id profile-id is-pinned is-pinned])
+  nil)
 
 ;; --- MUTATION: Rename Project
 
@@ -238,17 +238,17 @@
 (sv/defmethod ::rename-project
   {::doc/added "1.18"
    ::sm/params schema:rename-project
-   ::webhooks/event? true}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id name] :as params}]
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id id)
-    (let [project (db/get-by-id conn :project id ::sql/for-update true)]
-      (db/update! conn :project
-                  {:name name}
-                  {:id id})
-      (rph/with-meta (rph/wrap)
-        {::audit/props {:team-id (:team-id project)
-                        :prev-name (:name project)}}))))
+   ::webhooks/event? true
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id id name] :as params}]
+  (check-edition-permissions! conn profile-id id)
+  (let [project (db/get-by-id conn :project id ::sql/for-update true)]
+    (db/update! conn :project
+                {:name name}
+                {:id id})
+    (rph/with-meta (rph/wrap)
+      {::audit/props {:team-id (:team-id project)
+                      :prev-name (:name project)}})))
 
 ;; --- MUTATION: Delete Project
 
@@ -280,13 +280,13 @@
 (sv/defmethod ::delete-project
   {::doc/added "1.18"
    ::sm/params schema:delete-project
-   ::webhooks/event? true}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id] :as params}]
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id id)
-    (let [project (delete-project conn id)]
-      (rph/with-meta (rph/wrap)
-        {::audit/props {:team-id (:team-id project)
-                        :name (:name project)
-                        :created-at (:created-at project)
-                        :modified-at (:modified-at project)}}))))
+   ::webhooks/event? true
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id id] :as params}]
+  (check-edition-permissions! conn profile-id id)
+  (let [project (delete-project conn id)]
+    (rph/with-meta (rph/wrap)
+      {::audit/props {:team-id (:team-id project)
+                      :name (:name project)
+                      :created-at (:created-at project)
+                      :modified-at (:modified-at project)}})))

backend/src/app/rpc/commands/teams.clj

@@ -76,9 +76,10 @@
   (perms/make-check-fn has-read-permissions?))
 
 (defn decode-row
-  [{:keys [features] :as row}]
+  [{:keys [features subscription] :as row}]
   (cond-> row
-    (some? features) (assoc :features (db/decode-pgarray features #{}))))
+    (some? features) (assoc :features (db/decode-pgarray features #{}))
+    (some? subscription) (assoc :subscription (db/decode-transit-pgobject subscription))))
 
 ;; FIXME: move
 
@@ -113,29 +114,41 @@
 
 ;; --- Query: Teams
 
-(declare get-teams)
-
-(def ^:private schema:get-teams
-  [:map {:title "get-teams"}])
-
-(sv/defmethod ::get-teams
-  {::doc/added "1.17"
-   ::sm/params schema:get-teams}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id] :as params}]
-  (dm/with-open [conn (db/open pool)]
-    (get-teams conn profile-id)))
-
 (def sql:get-teams-with-permissions
-  "select t.*,
+  "SELECT t.*,
           tp.is_owner,
           tp.is_admin,
           tp.can_edit,
-          (t.id = ?) as is_default
-     from team_profile_rel as tp
-     join team as t on (t.id = tp.team_id)
-    where t.deleted_at is null
-      and tp.profile_id = ?
-    order by tp.created_at asc")
+          (t.id = ?) AS is_default
+     FROM team_profile_rel AS tp
+     JOIN team AS t ON (t.id = tp.team_id)
+    WHERE t.deleted_at IS null
+      AND tp.profile_id = ?
+    ORDER BY tp.created_at ASC")
+
+(def sql:get-teams-with-permissions-and-subscription
+  "SELECT t.*,
+          tp.is_owner,
+          tp.is_admin,
+          tp.can_edit,
+          (t.id = ?) AS is_default,
+
+          jsonb_build_object(
+            '~:type', COALESCE(p.props->'~:subscription'->>'~:type', 'professional'),
+            '~:status', CASE COALESCE(p.props->'~:subscription'->>'~:type', 'professional')
+                          WHEN 'professional' THEN 'active'
+                          ELSE COALESCE(p.props->'~:subscription'->>'~:status', 'incomplete')
+                       END
+          ) AS subscription
+     FROM team_profile_rel AS tp
+     JOIN team AS t ON (t.id = tp.team_id)
+     JOIN team_profile_rel AS tpr
+       ON (tpr.team_id = t.id AND tpr.is_owner IS true)
+     JOIN profile AS p
+       ON (tpr.profile_id = p.id)
+    WHERE t.deleted_at IS null
+      AND tp.profile_id = ?
+    ORDER BY tp.created_at ASC;")
 
 (defn process-permissions
   [team]
@@ -150,13 +163,52 @@
         (dissoc :is-owner :is-admin :can-edit)
         (assoc :permissions permissions))))
 
+(def ^:private
+  xform:process-teams
+  (comp
+   (map decode-row)
+   (map process-permissions)))
+
 (defn get-teams
   [conn profile-id]
-  (let [profile (profile/get-profile conn profile-id)]
-    (->> (db/exec! conn [sql:get-teams-with-permissions (:default-team-id profile) profile-id])
-         (map decode-row)
-         (map process-permissions)
-         (vec))))
+  (let [profile (profile/get-profile conn profile-id)
+        sql     (if (contains? cf/flags :subscriptions)
+                  sql:get-teams-with-permissions-and-subscription
+                  sql:get-teams-with-permissions)]
+
+    (->> (db/exec! conn [sql (:default-team-id profile) profile-id])
+         (into [] xform:process-teams))))
+
+(def ^:private schema:get-teams
+  [:map {:title "get-teams"}])
+
+(sv/defmethod ::get-teams
+  {::doc/added "1.17"
+   ::sm/params schema:get-teams}
+  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id] :as params}]
+  (dm/with-open [conn (db/open pool)]
+    (get-teams conn profile-id)))
+
+(def ^:private sql:get-owned-teams
+  "SELECT t.id, t.name,
+          (SELECT count(*) FROM team_profile_rel WHERE team_id=t.id) AS total_members
+     FROM team AS t
+     JOIN team_profile_rel AS tpr ON (tpr.team_id = t.id)
+    WHERE t.is_default IS false
+      AND tpr.is_owner IS true
+      AND tpr.profile_id = ?
+      AND t.deleted_at IS NULL")
+
+(defn- get-owned-teams
+  [cfg profile-id]
+  (->> (db/exec! cfg [sql:get-owned-teams profile-id])
+       (into [] (map decode-row))))
+
+(sv/defmethod ::get-owned-teams
+  {::doc/added "2.8.0"
+   ::sm/params schema:get-teams}
+  [cfg {:keys [::rpc/profile-id]}]
+  (get-owned-teams cfg profile-id))
 
 ;; --- Query: Team (by ID)
 
@@ -527,14 +579,14 @@
 
 (sv/defmethod ::update-team
   {::doc/added "1.17"
-   ::sm/params schema:update-team}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id name] :as params}]
-  (db/with-atomic [conn pool]
-    (check-edition-permissions! conn profile-id id)
-    (db/update! conn :team
-                {:name name}
-                {:id id})
-    nil))
+   ::sm/params schema:update-team
+   ::db/transaction true}
+  [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id id name]}]
+  (check-edition-permissions! conn profile-id id)
+  (db/update! conn :team
+              {:name name}
+              {:id id})
+  nil)
 
 
 ;; --- Mutation: Leave Team
@@ -592,10 +644,10 @@
 
 (sv/defmethod ::leave-team
   {::doc/added "1.17"
-   ::sm/params schema:leave-team}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id] :as params}]
-  (db/with-atomic [conn pool]
-    (leave-team conn (assoc params :profile-id profile-id))))
+   ::sm/params schema:leave-team
+   ::db/transaction true}
+  [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id] :as params}]
+  (leave-team conn (assoc params :profile-id profile-id)))
 
 ;; --- Mutation: Delete Team
 
@@ -627,16 +679,16 @@
 
 (sv/defmethod ::delete-team
   {::doc/added "1.17"
-   ::sm/params schema:delete-team}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id] :as params}]
-  (db/with-atomic [conn pool]
-    (let [perms (get-permissions conn profile-id id)]
-      (when-not (:is-owner perms)
-        (ex/raise :type :validation
-                  :code :only-owner-can-delete-team))
+   ::sm/params schema:delete-team
+   ::db/transaction true}
+  [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id id] :as params}]
+  (let [perms (get-permissions conn profile-id id)]
+    (when-not (:is-owner perms)
+      (ex/raise :type :validation
+                :code :only-owner-can-delete-team))
 
-      (delete-team conn id)
-      nil)))
+    (delete-team conn id)
+    nil))
 
 ;; --- Mutation: Team Update Role
 
@@ -714,31 +766,30 @@
 
 (sv/defmethod ::delete-team-member
   {::doc/added "1.17"
-   ::sm/params schema:delete-team-member}
-  [{:keys [::db/pool ::mbus/msgbus] :as cfg} {:keys [::rpc/profile-id team-id member-id] :as params}]
-  (db/with-atomic [conn pool]
-    (let [team  (get-team pool :profile-id profile-id :team-id team-id)
-          perms (get-permissions conn profile-id team-id)]
-      (when-not (or (:is-owner perms)
-                    (:is-admin perms))
-        (ex/raise :type :validation
-                  :code :insufficient-permissions))
+   ::sm/params schema:delete-team-member
+   ::db/transaction true}
+  [{:keys [::db/conn ::mbus/msgbus] :as cfg} {:keys [::rpc/profile-id team-id member-id] :as params}]
+  (let [team  (get-team conn :profile-id profile-id :team-id team-id)
+        perms (get-permissions conn profile-id team-id)]
+    (when-not (or (:is-owner perms)
+                  (:is-admin perms))
+      (ex/raise :type :validation
+                :code :insufficient-permissions))
 
-      (when (= member-id profile-id)
-        (ex/raise :type :validation
-                  :code :cant-remove-yourself))
+    (when (= member-id profile-id)
+      (ex/raise :type :validation
+                :code :cant-remove-yourself))
 
-      (db/delete! conn :team-profile-rel {:profile-id member-id
-                                          :team-id team-id})
+    (db/delete! conn :team-profile-rel {:profile-id member-id
+                                        :team-id team-id})
+    (mbus/pub! msgbus
+               :topic member-id
+               :message {:type :team-membership-change
+                         :change :removed
+                         :team-id team-id
+                         :team-name (:name team)})
 
-      (mbus/pub! msgbus
-                 :topic member-id
-                 :message {:type :team-membership-change
-                           :change :removed
-                           :team-id team-id
-                           :team-name (:name team)})
-
-      nil)))
+    nil))
 
 ;; --- Mutation: Update Team Photo
 
@@ -764,16 +815,16 @@
   (let [team  (get-team pool :profile-id profile-id :team-id team-id)
         photo (profile/upload-photo cfg params)]
 
-    (db/with-atomic [conn pool]
-      (check-admin-permissions! conn profile-id team-id)
-      ;; Mark object as touched for make it ellegible for tentative
-      ;; garbage collection.
-      (when-let [id (:photo-id team)]
-        (sto/touch-object! storage id))
+    (check-admin-permissions! pool profile-id team-id)
 
-      ;; Save new photo
-      (db/update! pool :team
-                  {:photo-id (:id photo)}
-                  {:id team-id})
+    ;; Mark object as touched for make it ellegible for tentative
+    ;; garbage collection.
+    (when-let [id (:photo-id team)]
+      (sto/touch-object! storage id))
 
-      (assoc team :photo-id (:id photo)))))
+    ;; Save new photo
+    (db/update! pool :team
+                {:photo-id (:id photo)}
+                {:id team-id})
+
+    (assoc team :photo-id (:id photo))))

backend/src/app/rpc/commands/teams_invitations.clj

@@ -408,20 +408,20 @@
 (sv/defmethod ::update-team-invitation-role
   {::doc/added "1.17"
    ::doc/module :teams
-   ::sm/params schema:update-team-invitation-role}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id team-id email role] :as params}]
-  (db/with-atomic [conn pool]
-    (let [perms (teams/get-permissions conn profile-id team-id)]
+   ::sm/params schema:update-team-invitation-role
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id team-id email role] :as params}]
+  (let [perms (teams/get-permissions conn profile-id team-id)]
 
-      (when-not (:is-admin perms)
-        (ex/raise :type :validation
-                  :code :insufficient-permissions))
+    (when-not (:is-admin perms)
+      (ex/raise :type :validation
+                :code :insufficient-permissions))
 
-      (db/update! conn :team-invitation
-                  {:role (name role) :updated-at (dt/now)}
-                  {:team-id team-id :email-to (profile/clean-email email)})
+    (db/update! conn :team-invitation
+                {:role (name role) :updated-at (dt/now)}
+                {:team-id team-id :email-to (profile/clean-email email)})
 
-      nil)))
+    nil))
 
 ;; --- Mutation: Delete invitation
 
@@ -432,20 +432,20 @@
 
 (sv/defmethod ::delete-team-invitation
   {::doc/added "1.17"
-   ::sm/params schema:delete-team-invition}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id team-id email] :as params}]
-  (db/with-atomic [conn pool]
-    (let [perms (teams/get-permissions conn profile-id team-id)]
+   ::sm/params schema:delete-team-invition
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id team-id email] :as params}]
+  (let [perms (teams/get-permissions conn profile-id team-id)]
 
-      (when-not (:is-admin perms)
-        (ex/raise :type :validation
-                  :code :insufficient-permissions))
+    (when-not (:is-admin perms)
+      (ex/raise :type :validation
+                :code :insufficient-permissions))
 
-      (let [invitation (db/delete! conn :team-invitation
-                                   {:team-id team-id
-                                    :email-to (profile/clean-email email)}
-                                   {::db/return-keys true})]
-        (rph/wrap nil {::audit/props {:invitation-id (:id invitation)}})))))
+    (let [invitation (db/delete! conn :team-invitation
+                                 {:team-id team-id
+                                  :email-to (profile/clean-email email)}
+                                 {::db/return-keys true})]
+      (rph/wrap nil {::audit/props {:invitation-id (:id invitation)}}))))
 
 
 ;; --- Mutation: Request Team Invitation

backend/src/app/rpc/commands/viewer.clj

@@ -78,7 +78,10 @@
                   :always
                   (update :data select-keys [:id :options :pages :pages-index :components]))
 
-        libs    (files/get-file-libraries conn file-id)
+        libs    (->> (files/get-file-libraries conn file-id)
+                     (mapv (fn [{:keys [id] :as lib}]
+                             (merge lib (files/get-file cfg id)))))
+
         links   (->> (db/query conn :share-link {:file-id file-id})
                      (mapv (fn [row]
                              (-> row

backend/src/app/rpc/commands/webhooks.clj

@@ -144,20 +144,20 @@
 
 (sv/defmethod ::delete-webhook
   {::doc/added "1.17"
-   ::sm/params schema:delete-webhook}
-  [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id id]}]
-  (db/with-atomic [conn pool]
-    (let [whook (-> (db/get conn :webhook {:id id}) decode-row)]
-      (check-webhook-edition-permissions! conn profile-id (:team-id whook) (:profile-id whook))
-      (db/delete! conn :webhook {:id id})
-      nil)))
+   ::sm/params schema:delete-webhook
+   ::db/transaction true}
+  [{:keys [::db/conn]} {:keys [::rpc/profile-id id]}]
+  (let [whook (-> (db/get conn :webhook {:id id}) decode-row)]
+    (check-webhook-edition-permissions! conn profile-id (:team-id whook) (:profile-id whook))
+    (db/delete! conn :webhook {:id id})
+    nil))
 
 ;; --- Query: Webhooks
 
 (def sql:get-webhooks
-  "SELECT id, uri, mtype, is_active, error_code, error_count, profile_id 
-     FROM webhook 
-    WHERE team_id = ? 
+  "SELECT id, uri, mtype, is_active, error_code, error_count, profile_id
+     FROM webhook
+    WHERE team_id = ?
     ORDER BY uri")
 
 (def ^:private schema:get-webhooks

backend/src/app/rpc/doc.clj

@@ -9,6 +9,7 @@
   (:require
    [app.common.data :as d]
    [app.common.exceptions :as ex]
+   [app.common.json :as json]
    [app.common.pprint :as pp]
    [app.common.schema :as sm]
    [app.common.schema.desc-js-like :as smdj]
@@ -19,7 +20,6 @@
    [app.http.sse :as-alias sse]
    [app.loggers.webhooks :as-alias webhooks]
    [app.rpc :as-alias rpc]
-   [app.util.json :as json]
    [app.util.services :as sv]
    [app.util.template :as tmpl]
    [clojure.java.io :as io]
@@ -86,7 +86,7 @@
     (fn [request]
       (let [params  (:query-params request)
             pstyle  (:type params "js")
-            context (assoc context :param-style pstyle)]
+            context (assoc @context :param-style pstyle)]
 
         {::yres/status 200
          ::yres/body (-> (io/resource "app/templates/api-doc.tmpl")
@@ -178,8 +178,7 @@
     (fn [_]
       {::yres/status 200
        ::yres/headers {"content-type" "application/json; charset=utf-8"}
-       ::yres/body (json/encode context)})
-
+       ::yres/body (json/encode @context)})
     (fn [_]
       {::yres/status 404})))
 
@@ -209,7 +208,7 @@
 
 (defmethod ig/init-key ::routes
   [_ {:keys [::rpc/methods] :as cfg}]
-  [(let [context (prepare-doc-context methods)]
+  [(let [context (delay (prepare-doc-context methods))]
      [["/_doc"
        {:handler (doc-handler context)
         :allowed-methods #{:get}}]
@@ -217,7 +216,7 @@
        {:handler (doc-handler context)
         :allowed-methods #{:get}}]])
 
-   (let [context (prepare-openapi-context methods)]
+   (let [context (delay (prepare-openapi-context methods))]
      [["/openapi"
        {:handler (openapi-handler)
         :allowed-methods #{:get}}]

backend/src/app/setup.clj

@@ -78,19 +78,19 @@
 
 (defmethod ig/init-key ::props
   [_ {:keys [::db/pool ::key] :as cfg}]
-  (db/with-atomic [conn pool]
-    (db/xact-lock! conn 0)
-    (when-not key
-      (l/warn :hint (str "using autogenerated secret-key, it will change on each restart and will invalidate "
-                         "all sessions on each restart, it is highly recommended setting up the "
-                         "PENPOT_SECRET_KEY environment variable")))
 
-    (let [secret (or key (generate-random-key))]
-      (-> (get-all-props conn)
-          (assoc :secret-key secret)
-          (assoc :tokens-key (keys/derive secret :salt "tokens"))
-          (update :instance-id handle-instance-id conn (db/read-only? pool))))))
+  (db/tx-run! cfg (fn [{:keys [::db/conn]}]
+                    (db/xact-lock! conn 0)
+                    (when-not key
+                      (l/warn :hint (str "using autogenerated secret-key, it will change on each restart and will invalidate "
+                                         "all sessions on each restart, it is highly recommended setting up the "
+                                         "PENPOT_SECRET_KEY environment variable")))
 
+                    (let [secret (or key (generate-random-key))]
+                      (-> (get-all-props conn)
+                          (assoc :secret-key secret)
+                          (assoc :tokens-key (keys/derive secret :salt "tokens"))
+                          (update :instance-id handle-instance-id conn (db/read-only? pool)))))))
 
 ;; FIXME
 (sm/register! ::props :any)

backend/src/app/srepl.clj

@@ -6,13 +6,17 @@
 
 (ns app.srepl
   "Server Repl."
+  (:refer-clojure :exclude [read-line])
   (:require
+   [app.common.exceptions :as ex]
+   [app.common.json :as json]
    [app.common.logging :as l]
    [app.config :as cf]
-   [app.srepl.cli]
+   [app.srepl.cli :as cli]
    [app.srepl.main]
-   [app.util.json :as json]
    [app.util.locks :as locks]
+   [app.util.time :as dt]
+   [clojure.core :as c]
    [clojure.core.server :as ccs]
    [clojure.main :as cm]
    [integrant.core :as ig]))
@@ -28,17 +32,80 @@
    :init repl-init
    :read ccs/repl-read))
 
+(defn- ex->data
+  [cause phase]
+  (let [data (ex-data cause)
+        explain (ex/explain data)]
+    (cond-> {:phase phase
+             :code (get data :code :unknown)
+             :type (get data :type :unknown)
+             :hint (or (get data :hint) (ex-message cause))}
+      (some? explain)
+      (assoc :explain explain))))
+
+(defn read-line
+  []
+  (if-let [line (c/read-line)]
+    (try
+      (l/dbg :hint "decode" :data line)
+      (json/decode line :key-fn json/read-kebab-key)
+      (catch Throwable _cause
+        (l/warn :hint "unable to decode data" :data line)
+        nil))
+    ::eof))
+
 (defn json-repl
   []
-  (let [out  *out*
-        lock (locks/create)]
-    (ccs/prepl *in*
-               (fn [m]
-                 (binding [*out* out,
-                           *flush-on-newline* true,
-                           *print-readably* true]
-                   (locks/locking lock
-                     (println (json/encode-str m))))))))
+  (let [lock (locks/create)
+        out  *out*
+
+        out-fn
+        (fn [m]
+          (locks/locking lock
+            (binding [*out* out]
+              (l/warn :hint "write" :data m)
+              (println (json/encode m :key-fn json/write-camel-key)))))
+
+        tapfn
+        (fn [val]
+          (out-fn {:tag :tap :val val}))]
+
+    (binding [*out* (PrintWriter-on #(out-fn {:tag :out :val %1}) nil true)
+              *err* (PrintWriter-on #(out-fn {:tag :err :val %1}) nil true)]
+      (try
+        (add-tap tapfn)
+        (loop []
+          (when (try
+                  (let [data   (read-line)
+                        tpoint (dt/tpoint)]
+
+                    (l/dbg :hint "received" :data (if (= data ::eof) "EOF" data))
+
+                    (try
+                      (when-not (= data ::eof)
+                        (when-not (nil? data)
+                          (let [result  (cli/exec data)
+                                elapsed (tpoint)]
+                            (l/warn :hint "result" :data result)
+                            (out-fn {:tag :ret
+                                     :val (if (instance? Throwable result)
+                                            (Throwable->map result)
+                                            result)
+                                     :elapsed (inst-ms elapsed)})))
+                        true)
+                      (catch Throwable cause
+                        (let [elapsed (tpoint)]
+                          (out-fn {:tag :ret
+                                   :err (ex->data cause :eval)
+                                   :elapsed (inst-ms elapsed)})
+                          true))))
+                  (catch Throwable cause
+                    (out-fn {:tag :ret
+                             :err (ex->data cause :read)})
+                    true))
+            (recur)))
+        (finally
+          (remove-tap tapfn))))))
 
 ;; --- State initialization
 

backend/src/app/srepl/cli.clj

@@ -9,14 +9,23 @@
   (:require
    [app.auth :as auth]
    [app.common.exceptions :as ex]
+   [app.common.schema :as sm]
    [app.common.uuid :as uuid]
    [app.db :as db]
    [app.rpc.commands.auth :as cmd.auth]
    [app.rpc.commands.profile :as cmd.profile]
-   [app.util.json :as json]
+   [app.setup :as-alias setup]
+   [app.tokens :as tokens]
    [app.util.time :as dt]
    [cuerdas.core :as str]))
 
+(defn coercer
+  [schema & {:as opts}]
+  (let [decode-fn (sm/decoder schema sm/json-transformer)
+        check-fn (sm/check-fn schema opts)]
+    (fn [data]
+      (-> data decode-fn check-fn))))
+
 (defn- get-current-system
   []
   (or (deref (requiring-resolve 'app.main/system))
@@ -24,88 +33,218 @@
 
 (defmulti ^:private exec-command ::cmd)
 
+(defmethod exec-command :default
+  [{:keys [::cmd]}]
+  (ex/raise :type :internal
+            :code :not-implemented
+            :hint (str/ffmt "command '%' not implemented" cmd)))
+
 (defn exec
   "Entry point with external tools integrations that uses PREPL
   interface for interacting with running penpot backend."
   [data]
-  (let [data (json/decode data)]
-    (-> {::cmd (keyword (:cmd data "default"))}
-        (merge (:params data))
-        (exec-command))))
+  (-> {::cmd (get data :cmd)}
+      (merge (:params data))
+      (exec-command)))
 
-(defmethod exec-command :create-profile
+(defmethod exec-command "create-profile"
   [{:keys [fullname email password is-active]
     :or {is-active true}}]
-  (when-let [system (get-current-system)]
-    (db/with-atomic [conn (:app.db/pool system)]
-      (let [password (cmd.profile/derive-password system password)
-            params   {:id (uuid/next)
-                      :email email
-                      :fullname fullname
-                      :is-active is-active
-                      :password password
-                      :props {}}]
-        (->> (cmd.auth/create-profile! conn params)
-             (cmd.auth/create-profile-rels! conn))))))
+  (some-> (get-current-system)
+          (db/tx-run!
+           (fn [{:keys [::db/conn] :as system}]
+             (let [password (cmd.profile/derive-password system password)
+                   params   {:id (uuid/next)
+                             :email email
+                             :fullname fullname
+                             :is-active is-active
+                             :password password
+                             :props {}}]
+               (->> (cmd.auth/create-profile! conn params)
+                    (cmd.auth/create-profile-rels! conn)))))))
 
-(defmethod exec-command :update-profile
+(defmethod exec-command "update-profile"
   [{:keys [fullname email password is-active]}]
-  (when-let [system (get-current-system)]
-    (db/with-atomic [conn (:app.db/pool system)]
-      (let [params (cond-> {}
-                     (some? fullname)
-                     (assoc :fullname fullname)
+  (some-> (get-current-system)
+          (db/tx-run!
+           (fn [{:keys [::db/conn] :as system}]
+             (let [params (cond-> {}
+                            (some? fullname)
+                            (assoc :fullname fullname)
 
-                     (some? password)
-                     (assoc :password (auth/derive-password password))
+                            (some? password)
+                            (assoc :password (auth/derive-password password))
 
-                     (some? is-active)
-                     (assoc :is-active is-active))]
-        (when (seq params)
-          (let [res (db/update! conn :profile
-                                params
-                                {:email email
-                                 :deleted-at nil})]
-            (pos? (db/get-update-count res))))))))
+                            (some? is-active)
+                            (assoc :is-active is-active))]
+               (when (seq params)
+                 (let [res (db/update! conn :profile
+                                       params
+                                       {:email email
+                                        :deleted-at nil})]
+                   (pos? (db/get-update-count res)))))))))
 
-(defmethod exec-command :delete-profile
+(defmethod exec-command "echo"
+  [params]
+  params)
+
+
+(defmethod exec-command "delete-profile"
   [{:keys [email soft]}]
   (when-not email
     (ex/raise :type :assertion
               :code :invalid-arguments
               :hint "email should be provided"))
 
-  (when-let [system (get-current-system)]
-    (db/with-atomic [conn (:app.db/pool system)]
+  (some-> (get-current-system)
+          (db/tx-run!
+           (fn [{:keys [::db/conn] :as system}]
+             (let [res (if soft
+                         (db/update! conn :profile
+                                     {:deleted-at (dt/now)}
+                                     {:email email :deleted-at nil})
+                         (db/delete! conn :profile
+                                     {:email email}))]
+               (pos? (db/get-update-count res)))))))
 
-      (let [res (if soft
-                  (db/update! conn :profile
-                              {:deleted-at (dt/now)}
-                              {:email email :deleted-at nil})
-                  (db/delete! conn :profile
-                              {:email email}))]
-        (pos? (db/get-update-count res))))))
-
-(defmethod exec-command :search-profile
+(defmethod exec-command "search-profile"
   [{:keys [email]}]
   (when-not email
     (ex/raise :type :assertion
               :code :invalid-arguments
               :hint "email should be provided"))
 
-  (when-let [system (get-current-system)]
-    (db/with-atomic [conn (:app.db/pool system)]
+  (some-> (get-current-system)
+          (db/tx-run!
+           (fn [{:keys [::db/conn] :as system}]
+             (let [sql (str "select email, fullname, created_at, deleted_at from profile "
+                            " where email similar to ? order by created_at desc limit 100")]
+               (db/exec! conn [sql email]))))))
 
-      (let [sql (str "select email, fullname, created_at, deleted_at from profile "
-                     " where email similar to ? order by created_at desc limit 100")]
-        (db/exec! conn [sql email])))))
-
-(defmethod exec-command :derive-password
+(defmethod exec-command "derive-password"
   [{:keys [password]}]
   (auth/derive-password password))
 
-(defmethod exec-command :default
-  [{:keys [::cmd]}]
-  (ex/raise :type :internal
-            :code :not-implemented
-            :hint (str/ffmt "command '%' not implemented" (name cmd))))
+(defmethod exec-command "authenticate"
+  [{:keys [token]}]
+  (when-let [system (get-current-system)]
+    (let [props  (get system ::setup/props)]
+      (tokens/verify props {:token token :iss "authentication"}))))
+
+(def ^:private schema:get-customer
+  [:map [:id ::sm/uuid]])
+
+(def coerce-get-customer-params
+  (coercer schema:get-customer
+           :type :validation
+           :hint "invalid data provided for `get-customer` rpc call"))
+
+(def sql:get-customer-slots
+  "WITH teams AS (
+    SELECT tpr.team_id AS id,
+           tpr.profile_id AS profile_id
+      FROM team_profile_rel AS tpr
+     WHERE tpr.is_owner IS true
+       AND tpr.profile_id = ?
+  ), teams_with_slots AS (
+    SELECT tpr.team_id AS id,
+           count(*) AS total
+      FROM team_profile_rel AS tpr
+     WHERE tpr.team_id IN (SELECT id FROM teams)
+       AND tpr.can_edit IS true
+     GROUP BY 1
+     ORDER BY 2
+  )
+  SELECT max(total) AS total FROM teams_with_slots;")
+
+(defn- get-customer-slots
+  [system profile-id]
+  (let [result (db/exec-one! system [sql:get-customer-slots profile-id])]
+    (:total result)))
+
+(defmethod exec-command "get-customer"
+  [params]
+  (when-let [system (get-current-system)]
+    (let [{:keys [id] :as params} (coerce-get-customer-params params)
+          {:keys [props] :as profile} (cmd.profile/get-profile system id)]
+      {:id (get profile :id)
+       :name (get profile :fullname)
+       :email (get profile :email)
+       :num-editors (get-customer-slots system id)
+       :subscription (get props :subscription)})))
+
+(def ^:private schema:customer-subscription
+  [:map {:title "CustomerSubscription"}
+   [:id ::sm/text]
+   [:customer-id ::sm/text]
+   [:type [:enum
+           "unlimited"
+           "professional"
+           "enterprise"]]
+   [:status [:enum
+             "active"
+             "canceled"
+             "incomplete"
+             "incomplete_expired"
+             "pass_due"
+             "paused"
+             "trialing"
+             "unpaid"]]
+
+   [:billing-period [:enum
+                     "month"
+                     "day"
+                     "week"
+                     "year"]]
+   [:quantity :int]
+   [:description [:maybe ::sm/text]]
+   [:created-at ::sm/timestamp]
+   [:start-date [:maybe ::sm/timestamp]]
+   [:ended-at [:maybe ::sm/timestamp]]
+   [:trial-end [:maybe ::sm/timestamp]]
+   [:trial-start [:maybe ::sm/timestamp]]
+   [:cancel-at [:maybe ::sm/timestamp]]
+   [:canceled-at [:maybe ::sm/timestamp]]
+
+   [:current-period-end ::sm/timestamp]
+   [:current-period-start ::sm/timestamp]
+   [:cancel-at-period-end :boolean]
+
+   [:cancellation-details
+    [:map {:title "CancellationDetails"}
+     [:comment [:maybe ::sm/text]]
+     [:reason [:maybe ::sm/text]]
+     [:feedback [:maybe
+                 [:enum
+                  "customer_service"
+                  "low_quality"
+                  "missing_feature"
+                  "other"
+                  "switched_service"
+                  "too_complex"
+                  "too_expensive"
+                  "unused"]]]]]])
+
+(def ^:private schema:update-customer-subscription
+  [:map
+   [:id ::sm/uuid]
+   [:subscription [:maybe schema:customer-subscription]]])
+
+(def coerce-update-customer-subscription-params
+  (coercer schema:update-customer-subscription
+           :type :validation
+           :hint "invalid data provided for `update-customer-subscription` rpc call"))
+
+(defmethod exec-command "update-customer-subscription"
+  [params]
+  (when-let [system (get-current-system)]
+    (let [{:keys [id subscription]} (coerce-update-customer-subscription-params params)
+          ;; FIXME: locking
+          {:keys [props] :as profile} (cmd.profile/get-profile system id)
+          props (assoc props :subscription subscription)]
+
+      (db/update! system :profile
+                  {:props (db/tjson props)}
+                  {:id id}
+                  {::db/return-keys false})
+      true)))

backend/src/app/srepl/components_v2.clj

@@ -1,306 +0,0 @@
-;; This Source Code Form is subject to the terms of the Mozilla Public
-;; License, v. 2.0. If a copy of the MPL was not distributed with this
-;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
-;;
-;; Copyright (c) KALEIDOS INC
-
-(ns app.srepl.components-v2
-  (:require
-   [app.common.fressian :as fres]
-   [app.common.logging :as l]
-   [app.db :as db]
-   [app.features.components-v2 :as feat]
-   [app.main :as main]
-   [app.srepl.helpers :as h]
-   [app.util.events :as events]
-   [app.util.time :as dt]
-   [app.worker :as-alias wrk]
-   [datoteka.fs :as fs]
-   [datoteka.io :as io]
-   [promesa.exec :as px]
-   [promesa.exec.semaphore :as ps]
-   [promesa.util :as pu]))
-
-(def ^:dynamic *scope* nil)
-(def ^:dynamic *semaphore* nil)
-
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;; PRIVATE HELPERS
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-
-(def ^:private sql:get-files-by-created-at
-  "SELECT id, features,
-          row_number() OVER (ORDER BY created_at DESC) AS rown
-     FROM file
-    WHERE deleted_at IS NULL
-    ORDER BY created_at DESC")
-
-(defn- get-files
-  [conn]
-  (->> (db/cursor conn [sql:get-files-by-created-at] {:chunk-size 500})
-       (map feat/decode-row)
-       (remove (fn [{:keys [features]}]
-                 (contains? features "components/v2")))))
-
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;; PUBLIC API
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-
-(defn migrate-file!
-  [file-id & {:keys [rollback? validate? label cache skip-on-graphic-error?]
-              :or {rollback? true
-                   validate? false
-                   skip-on-graphic-error? true}}]
-  (l/dbg :hint "migrate:start" :rollback rollback?)
-  (let [tpoint  (dt/tpoint)
-        file-id (h/parse-uuid file-id)]
-
-    (binding [feat/*stats* (atom {})
-              feat/*cache* cache]
-      (try
-        (-> (assoc main/system ::db/rollback rollback?)
-            (feat/migrate-file! file-id
-                                :validate? validate?
-                                :skip-on-graphic-error? skip-on-graphic-error?
-                                :label label))
-
-        (-> (deref feat/*stats*)
-            (assoc :elapsed (dt/format-duration (tpoint))))
-
-        (catch Throwable cause
-          (l/wrn :hint "migrate:error" :cause cause))
-
-        (finally
-          (let [elapsed (dt/format-duration (tpoint))]
-            (l/dbg :hint "migrate:end" :rollback rollback? :elapsed elapsed)))))))
-
-(defn migrate-team!
-  [team-id & {:keys [rollback? skip-on-graphic-error? validate? label cache]
-              :or {rollback? true
-                   validate? true
-                   skip-on-graphic-error? true}}]
-
-  (l/dbg :hint "migrate:start" :rollback rollback?)
-
-  (let [team-id (h/parse-uuid team-id)
-        stats   (atom {})
-        tpoint  (dt/tpoint)]
-
-    (binding [feat/*stats* stats
-              feat/*cache* cache]
-      (try
-        (-> (assoc main/system ::db/rollback rollback?)
-            (feat/migrate-team! team-id
-                                :label label
-                                :validate? validate?
-                                :skip-on-graphics-error? skip-on-graphic-error?))
-
-        (-> (deref feat/*stats*)
-            (assoc :elapsed (dt/format-duration (tpoint))))
-
-        (catch Throwable cause
-          (l/dbg :hint "migrate:error" :cause cause))
-
-        (finally
-          (let [elapsed (dt/format-duration (tpoint))]
-            (l/dbg :hint "migrate:end" :rollback rollback? :elapsed elapsed)))))))
-
-(defn migrate-files!
-  "A REPL helper for migrate all files.
-
-  This function starts multiple concurrent file migration processes
-  until thw maximum number of jobs is reached which by default has the
-  value of `1`. This is controled with the `:max-jobs` option.
-
-  If you want to run this on multiple machines you will need to specify
-  the total number of partitions and the current partition.
-
-  In order to get the report table populated, you will need to provide
-  a correct `:label`. That label is also used for persist a file
-  snaphot before continue with the migration."
-  [& {:keys [max-jobs max-items rollback? validate?
-             cache skip-on-graphic-error?
-             label partitions current-partition]
-      :or {validate? false
-           rollback? true
-           max-jobs 1
-           current-partition 1
-           skip-on-graphic-error? true
-           max-items Long/MAX_VALUE}}]
-
-  (when (int? partitions)
-    (when-not (int? current-partition)
-      (throw (IllegalArgumentException. "missing `current-partition` parameter")))
-    (when-not (<= 0 current-partition partitions)
-      (throw (IllegalArgumentException. "invalid value on `current-partition` parameter"))))
-
-  (let [stats     (atom {})
-        tpoint    (dt/tpoint)
-        factory   (px/thread-factory :virtual false :prefix "penpot/migration/")
-        executor  (px/cached-executor :factory factory)
-
-        sjobs     (ps/create :permits max-jobs)
-
-        migrate-file
-        (fn [file-id rown]
-          (try
-            (db/tx-run! (assoc main/system ::db/rollback rollback?)
-                        (fn [system]
-                          (db/exec-one! system ["SET LOCAL idle_in_transaction_session_timeout = 0"])
-                          (feat/migrate-file! system file-id
-                                              :rown rown
-                                              :label label
-                                              :validate? validate?
-                                              :skip-on-graphic-error? skip-on-graphic-error?)))
-
-            (catch Throwable cause
-              (l/wrn :hint "unexpected error on processing file (skiping)"
-                     :file-id (str file-id))
-
-              (events/tap :error
-                          (ex-info "unexpected error on processing file (skiping)"
-                                   {:file-id file-id}
-                                   cause))
-
-              (swap! stats update :errors (fnil inc 0)))
-
-            (finally
-              (ps/release! sjobs))))
-
-        process-file
-        (fn [{:keys [id rown]}]
-          (ps/acquire! sjobs)
-          (px/run! executor (partial migrate-file id rown)))]
-
-    (l/dbg :hint "migrate:start"
-           :label label
-           :rollback rollback?
-           :max-jobs max-jobs
-           :max-items max-items)
-
-    (binding [feat/*stats* stats
-              feat/*cache* cache]
-      (try
-        (db/tx-run! main/system
-                    (fn [{:keys [::db/conn] :as system}]
-                      (db/exec! conn ["SET LOCAL statement_timeout = 0"])
-                      (db/exec! conn ["SET LOCAL idle_in_transaction_session_timeout = 0"])
-
-                      (run! process-file
-                            (->> (get-files conn)
-                                 (filter (fn [{:keys [rown] :as row}]
-                                           (if (int? partitions)
-                                             (= current-partition (inc (mod rown partitions)))
-                                             true)))
-                                 (take max-items)))
-
-                      ;; Close and await tasks
-                      (pu/close! executor)))
-
-        (-> (deref stats)
-            (assoc :elapsed (dt/format-duration (tpoint))))
-
-        (catch Throwable cause
-          (l/dbg :hint "migrate:error" :cause cause)
-          (events/tap :error cause))
-
-        (finally
-          (let [elapsed (dt/format-duration (tpoint))]
-            (l/dbg :hint "migrate:end"
-                   :rollback rollback?
-                   :elapsed elapsed)))))))
-
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;; CACHE POPULATE
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-
-(def sql:sobjects-for-cache
-  "SELECT id,
-          row_number() OVER (ORDER BY created_at) AS index
-     FROM storage_object
-    WHERE (metadata->>'~:bucket' = 'file-media-object' OR
-           metadata->>'~:bucket' IS NULL)
-      AND metadata->>'~:content-type' = 'image/svg+xml'
-      AND deleted_at IS NULL
-      AND size < 1135899
-    ORDER BY created_at ASC")
-
-(defn populate-cache!
-  "A REPL helper for migrate all files.
-
-  This function starts multiple concurrent file migration processes
-  until thw maximum number of jobs is reached which by default has the
-  value of `1`. This is controled with the `:max-jobs` option.
-
-  If you want to run this on multiple machines you will need to specify
-  the total number of partitions and the current partition.
-
-  In order to get the report table populated, you will need to provide
-  a correct `:label`. That label is also used for persist a file
-  snaphot before continue with the migration."
-  [& {:keys [max-jobs] :or {max-jobs 1}}]
-
-  (let [tpoint    (dt/tpoint)
-
-        factory   (px/thread-factory :virtual false :prefix "penpot/cache/")
-        executor  (px/cached-executor :factory factory)
-
-        sjobs     (ps/create :permits max-jobs)
-
-        retrieve-sobject
-        (fn [id index]
-          (let [path   (feat/get-sobject-cache-path id)
-                parent (fs/parent path)]
-
-            (try
-              (when-not (fs/exists? parent)
-                (fs/create-dir parent))
-
-              (if (fs/exists? path)
-                (l/inf :hint "create cache entry" :status "exists" :index index :id (str id) :path (str path))
-                (let [svg-data (feat/get-optimized-svg id)]
-                  (with-open [^java.lang.AutoCloseable stream (io/output-stream path)]
-                    (let [writer (fres/writer stream)]
-                      (fres/write! writer svg-data)))
-
-                  (l/inf :hint "create cache entry" :status "created"
-                         :index index
-                         :id (str id)
-                         :path (str path))))
-
-              (catch Throwable cause
-                (l/wrn :hint "create cache entry"
-                       :status "error"
-                       :index index
-                       :id (str id)
-                       :path (str path)
-                       :cause cause))
-
-              (finally
-                (ps/release! sjobs)))))
-
-        process-sobject
-        (fn [{:keys [id index]}]
-          (ps/acquire! sjobs)
-          (px/run! executor (partial retrieve-sobject id index)))]
-
-    (l/dbg :hint "migrate:start"
-           :max-jobs max-jobs)
-
-    (try
-      (binding [feat/*system* main/system]
-        (run! process-sobject
-              (db/exec! main/system [sql:sobjects-for-cache]))
-
-        ;; Close and await tasks
-        (pu/close! executor))
-
-      {:elapsed (dt/format-duration (tpoint))}
-
-      (catch Throwable cause
-        (l/dbg :hint "populate:error" :cause cause))
-
-      (finally
-        (let [elapsed (dt/format-duration (tpoint))]
-          (l/dbg :hint "populate:end"
-                 :elapsed elapsed))))))

backend/src/app/srepl/fixes.clj

@@ -179,7 +179,7 @@
                       component-child (first component-children)]
                   (if (or (nil? child) (nil? component-child))
                     container
-                    (let [container (if (and (not (ctk/is-main-of? component-child child true))
+                    (let [container (if (and (not (ctk/is-main-of? component-child child))
                                              (nil? (ctk/get-swap-slot child))
                                              (ctk/instance-head? child))
                                       (let [slot (guess-swap-slot component-child component-container)]

backend/src/app/srepl/helpers.clj

@@ -10,9 +10,9 @@
   (:require
    [app.binfile.common :as bfc]
    [app.common.data :as d]
+   [app.common.files.migrations :as fmg]
    [app.common.files.validate :as cfv]
    [app.db :as db]
-   [app.features.components-v2 :as feat.comp-v2]
    [app.main :as main]
    [app.rpc.commands.files :as files]
    [app.rpc.commands.files-snapshot :as fsnap]
@@ -61,6 +61,27 @@
                 {:id id})
     team))
 
+(def ^:private sql:get-and-lock-team-files
+  "SELECT f.id
+     FROM file AS f
+     JOIN project AS p ON (p.id = f.project_id)
+    WHERE p.team_id = ?
+      AND p.deleted_at IS NULL
+      AND f.deleted_at IS NULL
+      FOR UPDATE")
+
+(defn get-team
+  [conn team-id]
+  (-> (db/get conn :team {:id team-id}
+              {::db/remove-deleted false
+               ::db/check-deleted false})
+      (update :features db/decode-pgarray #{})))
+
+(defn get-and-lock-team-files
+  [conn team-id]
+  (transduce (map :id) conj []
+             (db/plan conn [sql:get-and-lock-team-files team-id])))
+
 (defn reset-file-data!
   "Hardcode replace of the data of one file."
   [system id data]
@@ -95,7 +116,7 @@
 (defn take-team-snapshot!
   [system team-id label]
   (let [conn (db/get-connection system)]
-    (->> (feat.comp-v2/get-and-lock-team-files conn team-id)
+    (->> (get-and-lock-team-files conn team-id)
          (reduce (fn [result file-id]
                    (let [file (fsnap/get-file-snapshots system file-id)]
                      (fsnap/create-file-snapshot! system file
@@ -107,19 +128,16 @@
 (defn restore-team-snapshot!
   [system team-id label]
   (let [conn (db/get-connection system)
-        ids  (->> (feat.comp-v2/get-and-lock-team-files conn team-id)
+        ids  (->> (get-and-lock-team-files conn team-id)
                   (into #{}))
 
         snap (search-file-snapshots conn ids label)
 
-        ids' (into #{} (map :file-id) snap)
-        team (-> (feat.comp-v2/get-team conn team-id)
-                 (update :features disj "components/v2"))]
+        ids' (into #{} (map :file-id) snap)]
 
     (when (not= ids ids')
       (throw (RuntimeException. "no uniform snapshot available")))
 
-    (feat.comp-v2/update-team! conn team)
     (reduce (fn [result {:keys [file-id id]}]
               (fsnap/restore-file-snapshot! system file-id id)
               (inc result))
@@ -142,7 +160,9 @@
                   (update-fn file opts)))]
 
     (when (and (some? file')
-               (not (identical? file file')))
+               (or (fmg/migrated? file)
+                   (not (identical? file file'))))
+
       (when validate?
         (cfv/validate-file-schema! file'))
 

backend/src/app/srepl/main.clj

@@ -22,7 +22,6 @@
    [app.config :as cf]
    [app.db :as db]
    [app.db.sql :as-alias sql]
-   [app.features.components-v2 :as feat.comp-v2]
    [app.features.fdata :as feat.fdata]
    [app.loggers.audit :as audit]
    [app.main :as main]
@@ -101,38 +100,46 @@
   "Mark the profile blocked and removes all the http sessiones
   associated with the profile-id."
   [email]
-  (db/with-atomic [conn (:app.db/pool main/system)]
-    (when-let [profile (db/get* conn :profile
-                                {:email (str/lower email)}
-                                {:columns [:id :email]})]
-      (when-not (:is-blocked profile)
-        (db/update! conn :profile {:is-active true} {:id (:id profile)})
-        :activated))))
+  (some-> main/system
+          (db/tx-run!
+           (fn [{:keys [::db/conn] :as system}]
+             (when-let [profile (db/get* conn :profile
+                                         {:email (str/lower email)}
+                                         {:columns [:id :email]})]
+               (when-not (:is-blocked profile)
+                 (db/update! conn :profile {:is-active true} {:id (:id profile)})
+                 :activated))))))
 
 (defn mark-profile-as-blocked!
   "Mark the profile blocked and removes all the http sessiones
   associated with the profile-id."
   [email]
-  (db/with-atomic [conn (:app.db/pool main/system)]
-    (when-let [profile (db/get* conn :profile
-                                {:email (str/lower email)}
-                                {:columns [:id :email]})]
-      (when-not (:is-blocked profile)
-        (db/update! conn :profile {:is-blocked true} {:id (:id profile)})
-        (db/delete! conn :http-session {:profile-id (:id profile)})
-        :blocked))))
+  (some-> main/system
+          (db/tx-run!
+           (fn [{:keys [::db/conn] :as system}]
+             (when-let [profile (db/get* conn :profile
+                                         {:email (str/lower email)}
+                                         {:columns [:id :email]})]
+               (when-not (:is-blocked profile)
+                 (db/update! conn :profile {:is-blocked true} {:id (:id profile)})
+                 (db/delete! conn :http-session {:profile-id (:id profile)})
+                 :blocked))))))
 
 (defn reset-password!
   "Reset a password to a specific one for a concrete user or all users
   if email is `:all` keyword."
   [& {:keys [email password] :or {password "123123"} :as params}]
-  (us/verify! (contains? params :email) "`email` parameter is mandatory")
-  (db/with-atomic [conn (:app.db/pool main/system)]
-    (let [password (derive-password password)]
-      (if (= email :all)
-        (db/exec! conn ["update profile set password=?" password])
-        (let [email (str/lower email)]
-          (db/exec! conn ["update profile set password=? where email=?" password email]))))))
+  (when-not email
+    (throw (IllegalArgumentException. "email is mandatory")))
+
+  (some-> main/system
+          (db/tx-run!
+           (fn [{:keys [::db/conn] :as system}]
+             (let [password (derive-password password)]
+               (if (= email :all)
+                 (db/exec! conn ["update profile set password=?" password])
+                 (let [email (str/lower email)]
+                   (db/exec! conn ["update profile set password=? where email=?" password email]))))))))
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; FEATURES
@@ -148,14 +155,18 @@
   [file-id & {:as opts}]
   (process-file! file-id feat.fdata/enable-pointer-map opts))
 
+(defn enable-path-data-feature-on-file!
+  [file-id & {:as opts}]
+  (process-file! file-id feat.fdata/enable-path-data opts))
+
 (defn enable-storage-features-on-file!
   [file-id & {:as opts}]
   (enable-objects-map-feature-on-file! file-id opts)
   (enable-pointer-map-feature-on-file! file-id opts))
 
 (defn enable-team-feature!
-  [team-id feature]
-  (when-not (contains? cfeat/supported-features feature)
+  [team-id feature & {:keys [skip-check] :or {skip-check false}}]
+  (when (and (not skip-check) (not (contains? cfeat/supported-features feature)))
     (ex/raise :type :assertion
               :code :feature-not-supported
               :hint (str "feature '" feature "' not supported")))
@@ -173,9 +184,8 @@
                       :enabled))))))
 
 (defn disable-team-feature!
-  [team-id feature]
-
-  (when-not (contains? cfeat/supported-features feature)
+  [team-id feature & {:keys [skip-check] :or {skip-check false}}]
+  (when (and (not skip-check) (not (contains? cfeat/supported-features feature)))
     (ex/raise :type :assertion
               :code :feature-not-supported
               :hint (str "feature '" feature "' not supported")))
@@ -202,100 +212,116 @@
   This method allows send flash notifications to specified target destinations.
   The message can be a free text or a preconfigured one.
 
-  The destination can be: all, profile-id, team-id, or a coll of them."
-  [{:keys [::mbus/msgbus ::db/pool]} & {:keys [dest code message level]
-                                        :or {code :generic level :info}
-                                        :as params}]
+  The destination can be: all, profile-id, team-id, or a coll of them.
+  It also can be:
+
+  {:email \"some@example.com\"}
+  [[:email \"some@example.com\"], ...]
+
+  Command examples:
+
+  (notify! :dest :all :code :maintenance)
+  (notify! :dest :all :code :upgrade-version)
+  "
+  [& {:keys [dest code message level]
+      :or {code :generic level :info}
+      :as params}]
 
   (when-not (contains? #{:success :error :info :warning} level)
     (ex/raise :type :assertion
               :code :incorrect-level
               :hint (str "level '" level "' not supported")))
 
-  (letfn [(send [dest]
-            (l/inf :hint "sending notification" :dest (str dest))
-            (let [message {:type :notification
-                           :code code
-                           :level level
-                           :version (:full cf/version)
-                           :subs-id dest
-                           :message message}
-                  message (->> (dissoc params :dest :code :message :level)
-                               (merge message))]
-              (mbus/pub! msgbus
-                         :topic (str dest)
-                         :message message)))
+  (let [{:keys [::mbus/msgbus ::db/pool]} main/system
 
-          (resolve-profile [email]
-            (some-> (db/get* pool :profile {:email (str/lower email)} {:columns [:id]}) :id vector))
+        send
+        (fn [dest]
+          (l/inf :hint "sending notification" :dest (str dest))
+          (let [message {:type :notification
+                         :code code
+                         :level level
+                         :version (:full cf/version)
+                         :subs-id dest
+                         :message message}
+                message (->> (dissoc params :dest :code :message :level)
+                             (merge message))]
+            (mbus/pub! msgbus
+                       :topic dest
+                       :message message)))
 
-          (resolve-team [team-id]
-            (->> (db/query pool :team-profile-rel
-                           {:team-id team-id}
-                           {:columns [:profile-id]})
-                 (map :profile-id)))
+        resolve-profile
+        (fn [email]
+          (some-> (db/get* pool :profile {:email (str/lower email)} {:columns [:id]}) :id vector))
 
-          (resolve-dest [dest]
-            (cond
-              (= :all dest)
-              [uuid/zero]
+        resolve-team
+        (fn [team-id]
+          (->> (db/query pool :team-profile-rel
+                         {:team-id team-id}
+                         {:columns [:profile-id]})
+               (map :profile-id)))
 
-              (uuid? dest)
-              [dest]
+        resolve-dest
+        (fn resolve-dest [dest]
+          (cond
+            (= :all dest)
+            [uuid/zero]
 
-              (string? dest)
-              (some-> dest h/parse-uuid resolve-dest)
+            (uuid? dest)
+            [dest]
 
-              (nil? dest)
-              (resolve-dest uuid/zero)
+            (string? dest)
+            (some-> dest h/parse-uuid resolve-dest)
 
-              (map? dest)
-              (sequence (comp
-                         (map vec)
-                         (mapcat resolve-dest))
-                        dest)
+            (nil? dest)
+            [uuid/zero]
 
-              (and (vector? dest)
-                   (every? vector? dest))
-              (sequence (comp
-                         (map vec)
-                         (mapcat resolve-dest))
-                        dest)
+            (map? dest)
+            (sequence (comp
+                       (map vec)
+                       (mapcat resolve-dest))
+                      dest)
 
-              (and (vector? dest)
-                   (keyword? (first dest)))
-              (let [[op param] dest]
+            (and (vector? dest)
+                 (every? vector? dest))
+            (sequence (comp
+                       (map vec)
+                       (mapcat resolve-dest))
+                      dest)
+
+            (and (vector? dest)
+                 (keyword? (first dest)))
+            (let [[op param] dest]
+              (cond
+                (= op :email)
                 (cond
-                  (= op :email)
-                  (cond
-                    (and (coll? param)
-                         (every? string? param))
-                    (sequence (comp
-                               (keep resolve-profile)
-                               (mapcat identity))
-                              param)
+                  (and (coll? param)
+                       (every? string? param))
+                  (sequence (comp
+                             (keep resolve-profile)
+                             (mapcat identity))
+                            param)
 
-                    (string? param)
-                    (resolve-profile param))
+                  (string? param)
+                  (resolve-profile param))
 
-                  (= op :team-id)
-                  (cond
-                    (coll? param)
-                    (sequence (comp
-                               (mapcat resolve-team)
-                               (keep h/parse-uuid))
-                              param)
+                (= op :team-id)
+                (cond
+                  (coll? param)
+                  (sequence (comp
+                             (mapcat resolve-team)
+                             (keep h/parse-uuid))
+                            param)
 
-                    (uuid? param)
-                    (resolve-team param)
+                  (uuid? param)
+                  (resolve-team param)
 
-                    (string? param)
-                    (some-> param h/parse-uuid resolve-team))
+                  (string? param)
+                  (some-> param h/parse-uuid resolve-team))
 
-                  (= op :profile-id)
-                  (if (coll? param)
-                    (sequence (keep h/parse-uuid) param)
-                    (resolve-dest param))))))]
+                (= op :profile-id)
+                (if (coll? param)
+                  (sequence (keep h/parse-uuid) param)
+                  (resolve-dest param))))))]
 
     (->> (resolve-dest dest)
          (filter some?)
@@ -314,14 +340,23 @@
     (db/tx-run! main/system fsnap/create-file-snapshot! {:file-id file-id :label label})))
 
 (defn restore-file-snapshot!
-  [file-id label]
-  (let [file-id (h/parse-uuid file-id)]
+  [file-id & {:keys [label id]}]
+  (let [file-id     (h/parse-uuid file-id)
+        snapshot-id (some-> id h/parse-uuid)]
     (db/tx-run! main/system
                 (fn [{:keys [::db/conn] :as system}]
-                  (when-let [snapshot (->> (h/search-file-snapshots conn #{file-id} label)
-                                           (map :id)
-                                           (first))]
-                    (fsnap/restore-file-snapshot! system file-id (:id snapshot)))))))
+                  (cond
+                    (uuid? snapshot-id)
+                    (fsnap/restore-file-snapshot! system file-id snapshot-id)
+
+                    (string? label)
+                    (->> (h/search-file-snapshots conn #{file-id} label)
+                         (map :id)
+                         (first)
+                         (fsnap/restore-file-snapshot! system file-id))
+
+                    :else
+                    (throw (ex-info "snapshot id or label should be provided" {})))))))
 
 (defn list-file-snapshots!
   [file-id & {:as _}]
@@ -384,10 +419,12 @@
   "Apply a function to the file. Optionally save the changes or not.
   The function receives the decoded and migrated file data."
   [file-id update-fn & {:keys [rollback?] :or {rollback? true} :as opts}]
-  (db/tx-run! (assoc main/system ::db/rollback rollback?)
-              (fn [system]
-                (binding [h/*system* system]
-                  (h/process-file! system file-id update-fn opts)))))
+  (let [file-id (h/parse-uuid file-id)]
+    (db/tx-run! (assoc main/system ::db/rollback rollback?)
+                (fn [system]
+                  (binding [h/*system* system
+                            db/*conn* (db/get-connection system)]
+                    (h/process-file! system file-id update-fn opts))))))
 
 (defn process-team-files!
   "Apply a function to each file of the specified team."
@@ -399,8 +436,9 @@
                   (when (string? label)
                     (h/take-team-snapshot! system team-id label))
 
-                  (binding [h/*system* system]
-                    (->> (feat.comp-v2/get-and-lock-team-files conn team-id)
+                  (binding [h/*system* system
+                            db/*conn* (db/get-connection system)]
+                    (->> (h/get-and-lock-team-files conn team-id)
                          (reduce (fn [result file-id]
                                    (if (h/process-file! system file-id update-fn opts)
                                      (inc result)

backend/src/app/tasks/file_gc.clj

@@ -10,6 +10,7 @@
   file is eligible to be garbage collected after some period of
   inactivity (the default threshold is 72h)."
   (:require
+   [app.binfile.cleaner :as bfl]
    [app.binfile.common :as bfc]
    [app.common.files.helpers :as cfh]
    [app.common.files.validate :as cfv]
@@ -258,6 +259,7 @@
   (if-let [file (get-file cfg file-id)]
     (let [file (->> file
                     (bfc/decode-file cfg)
+                    (bfl/clean-file)
                     (clean-media! cfg)
                     (clean-fragments! cfg))
           file (assoc file :has-media-trimmed true)]

backend/src/app/tasks/tasks_gc.clj

@@ -26,18 +26,14 @@
   {k (assoc v ::min-age (cf/get-deletion-delay))})
 
 (defmethod ig/init-key ::handler
-  [_ {:keys [::db/pool ::min-age] :as cfg}]
+  [_ {:keys [::min-age] :as cfg}]
   (fn [{:keys [props] :as task}]
     (let [min-age (or (:min-age props) min-age)]
-      (db/with-atomic [conn pool]
-        (let [interval (db/interval min-age)
-              result   (db/exec-one! conn [sql:delete-completed-tasks interval])
-              result   (db/get-update-count result)]
-
-          (l/debug :hint "task finished" :total result)
-
-          (when (:rollback? props)
-            (db/rollback! conn))
-
-          result)))))
-
+      (-> cfg
+          (assoc ::db/rollback (:rollback? props))
+          (db/tx-run! (fn [{:keys [::db/conn]}]
+                        (let [interval (db/interval min-age)
+                              result   (db/exec-one! conn [sql:delete-completed-tasks interval])
+                              result   (db/get-update-count result)]
+                          (l/debug :hint "task finished" :total result)
+                          result)))))))

backend/src/app/util/events.clj

@@ -10,7 +10,6 @@
   to them. Mainly used in http.sse for progress reporting."
   (:refer-clojure :exclude [tap run!])
   (:require
-   [app.common.data.macros :as dm]
    [app.common.exceptions :as ex]
    [app.common.logging :as l]
    [promesa.exec :as px]
@@ -18,33 +17,30 @@
 
 (def ^:dynamic *channel* nil)
 
-(defn channel
-  []
-  (sp/chan :buf 32))
-
 (defn tap
-  [type data]
-  (when-let [channel *channel*]
-    (sp/put! channel [type data])
-    nil))
+  ([type data]
+   (when-let [channel *channel*]
+     (sp/put! channel [type data])
+     nil))
+  ([channel type data]
+   (when channel
+     (sp/put! channel [type data])
+     nil)))
 
 (defn start-listener
-  [on-event on-close]
-
-  (dm/assert!
-   "expected active events channel"
-   (sp/chan? *channel*))
+  [channel on-event on-close]
+  (assert (sp/chan? channel) "expected active events channel")
 
   (px/thread
     {:virtual true}
     (try
       (loop []
-        (when-let [event (sp/take! *channel*)]
+        (when-let [event (sp/take! channel)]
           (let [result (ex/try! (on-event event))]
             (if (ex/exception? result)
               (do
                 (l/wrn :hint "unexpected exception" :cause result)
-                (sp/close! *channel*))
+                (sp/close! channel))
               (recur)))))
       (finally
         (on-close)))))
@@ -55,7 +51,7 @@
   [f on-event]
 
   (binding [*channel* (sp/chan :buf 32)]
-    (let [listener (start-listener on-event (constantly nil))]
+    (let [listener (start-listener *channel* on-event (constantly nil))]
       (try
         (f)
         (finally

backend/src/app/worker/dispatcher.clj

@@ -71,11 +71,12 @@
 
           (run-batch! [rconn]
             (try
-              (db/with-atomic [conn pool]
-                (if-let [tasks (get-tasks conn)]
-                  (->> (group-by :queue tasks)
-                       (run! (partial push-tasks! conn rconn)))
-                  (px/sleep (::wait-duration cfg))))
+              (db/tx-run! cfg (fn [{:keys [::db/conn]}]
+                                (if-let [tasks (get-tasks conn)]
+                                  (->> (group-by :queue tasks)
+                                       (run! (partial push-tasks! conn rconn)))
+                                  ;; FIXME: this sleep should be outside the transaction
+                                  (px/sleep (::wait-duration cfg)))))
               (catch InterruptedException cause
                 (throw cause))
               (catch Exception cause

backend/src/app/worker/runner.clj

@@ -24,6 +24,33 @@
 
 (set! *warn-on-reflection* true)
 
+(def schema:task
+  [:map {:title "Task"}
+   [:id ::sm/uuid]
+   [:queue :string]
+   [:name :string]
+   [:created-at ::sm/inst]
+   [:modified-at ::sm/inst]
+   [:scheduled-at {:optional true} ::sm/inst]
+   [:completed-at {:optional true} ::sm/inst]
+   [:error {:optional true} :string]
+   [:max-retries :int]
+   [:retry-num :int]
+   [:priority :int]
+   [:status [:enum "scheduled" "completed" "new" "retry" "failed"]]
+   [:label {:optional true} :string]
+   [:props :map]])
+
+(def schema:result
+  [:map {:title "TaskResult"}
+   [:status [:enum "retry" "failed" "completed"]]
+   [:error {:optional true} [:fn ex/exception?]]
+   [:inc-by {:optional true} :int]
+   [:delay {:optional true} :int]])
+
+(def valid-task-result?
+  (sm/validator schema:result))
+
 (defn- decode-task-row
   [{:keys [props] :as row}]
   (cond-> row
@@ -51,10 +78,11 @@
            :retry (:retry-num task))
     (let [tpoint  (dt/tpoint)
           task-fn (wrk/get-task registry (:name task))
-          result  (if task-fn
-                    (task-fn task)
-                    {:status :completed :task task})
-          elapsed (dt/format-duration (tpoint))]
+          result  (when task-fn (task-fn task))
+          elapsed (dt/format-duration (tpoint))
+          result  (if (valid-task-result? result)
+                    result
+                    {:status "completed"})]
 
       (when-not task-fn
         (l/wrn :hint "no task handler found" :name (:name task)))
@@ -76,7 +104,7 @@
         (if (and (< (:retry-num task)
                     (:max-retries task))
                  (= ::retry (:type edata)))
-          (cond-> {:status :retry :task task :error cause}
+          (cond-> {:status "retry" :error cause}
             (dt/duration? (:delay edata))
             (assoc :delay (:delay edata))
 
@@ -87,8 +115,8 @@
                    ::l/context (get-error-context cause task)
                    :cause cause)
             (if (>= (:retry-num task) (:max-retries task))
-              {:status :failed :task task :error cause}
-              {:status :retry :task task :error cause})))))))
+              {:status "failed" :error cause}
+              {:status "retry" :error cause})))))))
 
 (defn- run-task!
   [{:keys [::id ::timeout] :as cfg} task-id]
@@ -116,12 +144,17 @@
              :task-id task-id)
 
       :else
-      (run-task cfg task))))
+      (let [result (run-task cfg task)]
+        (with-meta result
+          {::task task})))))
 
 (defn- run-worker-loop!
   [{:keys [::db/pool ::rds/rconn ::timeout ::queue] :as cfg}]
-  (letfn [(handle-task-retry [{:keys [task error inc-by delay] :or {inc-by 1 delay 1000}}]
-            (let [explain (ex-message error)
+  (letfn [(handle-task-retry [{:keys [error inc-by delay] :or {inc-by 1 delay 1000} :as result}]
+            (let [explain (if (ex/exception? error)
+                            (ex-message error)
+                            (str error))
+                  task    (-> result meta ::task)
                   nretry  (+ (:retry-num task) inc-by)
                   now     (dt/now)
                   delay   (->> (iterate #(* % 2) delay) (take nretry) (last))]
@@ -134,8 +167,9 @@
                           {:id (:id task)})
               nil))
 
-          (handle-task-failure [{:keys [task error]}]
-            (let [explain (ex-message error)]
+          (handle-task-failure [{:keys [error] :as result}]
+            (let [task    (-> result meta ::task)
+                  explain (ex-message error)]
               (db/update! pool :task
                           {:error explain
                            :modified-at (dt/now)
@@ -143,8 +177,9 @@
                           {:id (:id task)})
               nil))
 
-          (handle-task-completion [{:keys [task]}]
-            (let [now (dt/now)]
+          (handle-task-completion [result]
+            (let [task (-> result meta ::task)
+                  now  (dt/now)]
               (db/update! pool :task
                           {:completed-at now
                            :modified-at now
@@ -168,10 +203,11 @@
           (process-result [{:keys [status] :as result}]
             (ex/try!
              (case status
-               :retry (handle-task-retry result)
-               :failed (handle-task-failure result)
-               :completed (handle-task-completion result)
-               nil)))
+               "retry"     (handle-task-retry result)
+               "failed"    (handle-task-failure result)
+               "completed" (handle-task-completion result)
+               (throw (IllegalArgumentException.
+                       (str "invalid status received: " status))))))
 
           (run-task-loop [task-id]
             (loop [result (run-task! cfg task-id)]

backend/test/backend_tests/helpers.clj

@@ -138,14 +138,13 @@
                  "  FROM information_schema.tables "
                  " WHERE table_schema = 'public' "
                  "   AND table_name != 'migrations';")]
-    (db/with-atomic [conn *pool*]
-      (db/exec-one! conn ["SET CONSTRAINTS ALL DEFERRED"])
-      (db/exec-one! conn ["SET LOCAL rules.deletion_protection TO off"])
-      (let [result (->> (db/exec! conn [sql])
-                        (map :table-name))]
-        (doseq [table result]
-          (db/exec! conn [(str "delete from " table ";")]))))
-
+    (db/transact! *pool* (fn [conn]
+                           (db/exec-one! conn ["SET CONSTRAINTS ALL DEFERRED"])
+                           (db/exec-one! conn ["SET LOCAL rules.deletion_protection TO off"])
+                           (let [result (->> (db/exec! conn [sql])
+                                             (map :table-name))]
+                             (doseq [table result]
+                               (db/exec! conn [(str "delete from " table ";")])))))
     (next)))
 
 (defn clean-storage

backend/test/backend_tests/http_middleware_access_token_test.clj

@@ -20,45 +20,33 @@
 (t/use-fixtures :each th/database-reset)
 
 (t/deftest soft-auth-middleware
-  (db/with-atomic [conn (::db/pool th/*system*)]
-    (let [profile (th/create-profile* 1)
-          system  (-> th/*system*
-                      (assoc ::db/conn conn)
-                      (assoc ::main/props (:app.setup/props th/*system*)))
+  (let [profile (th/create-profile* 1)
+        token   (db/tx-run! th/*system* app.rpc.commands.access-token/create-access-token (:id profile) "test" nil)
 
-          token   (app.rpc.commands.access-token/create-access-token
-                   system (:id profile) "test" nil)
+        request (volatile! nil)
+        handler (#'app.http.access-token/wrap-soft-auth
+                 (fn [req] (vreset! request req))
+                 th/*system*)]
 
-          request (volatile! nil)
-          handler (#'app.http.access-token/wrap-soft-auth
-                   (fn [req] (vreset! request req))
-                   system)]
+    (with-mocks [m1 {:target 'app.http.access-token/get-token
+                     :return nil}]
+      (handler {})
+      (t/is (= {} @request)))
 
-      (with-mocks [m1 {:target 'app.http.access-token/get-token
-                       :return nil}]
-        (handler {})
-        (t/is (= {} @request)))
+    (with-mocks [m1 {:target 'app.http.access-token/get-token
+                     :return (:token token)}]
+      (handler {})
 
-      (with-mocks [m1 {:target 'app.http.access-token/get-token
-                       :return (:token token)}]
-        (handler {})
-
-        (let [token-id (get @request :app.http.access-token/id)]
-          (t/is (= token-id (:id token))))))))
+      (let [token-id (get @request :app.http.access-token/id)]
+        (t/is (= token-id (:id token)))))))
 
 (t/deftest authz-middleware
   (let [profile (th/create-profile* 1)
-        system  (assoc th/*system* ::main/props (:app.setup/props th/*system*))
-
-        token   (db/with-atomic [conn (::db/pool th/*system*)]
-                  (let [system (assoc system ::db/conn conn)]
-                    (app.rpc.commands.access-token/create-access-token
-                     system (:id profile) "test" nil)))
-
+        token   (db/tx-run! th/*system* app.rpc.commands.access-token/create-access-token (:id profile) "test" nil)
         request (volatile! {})
         handler (#'app.http.access-token/wrap-authz
                  (fn [req] (vreset! request req))
-                 system)]
+                 th/*system*)]
 
     (handler nil)
     (t/is (nil? @request))

backend/test/backend_tests/rpc_file_test.clj

@@ -1712,6 +1712,7 @@
         [{:fill-image
           {:id (:id fmedia)
            :name "test"
+           :mtype "image/jpeg"
            :width 200
            :height 200}}]]
 

backend/test/backend_tests/rpc_media_test.clj

@@ -48,7 +48,7 @@
         (t/is (sto/object? mobj1))
         (t/is (sto/object? mobj2))
         (t/is (= 122785 (:size mobj1)))
-        (t/is (= 3302 (:size mobj2)))))))
+        (t/is (= 3299 (:size mobj2)))))))
 
 (t/deftest media-object-upload
   (let [prof   (th/create-profile* 1)
@@ -85,7 +85,7 @@
         (t/is (sto/object? mobj1))
         (t/is (sto/object? mobj2))
         (t/is (= 312043 (:size mobj1)))
-        (t/is (= 3887   (:size mobj2)))))))
+        (t/is (= 3901   (:size mobj2)))))))
 
 
 (t/deftest media-object-upload-idempotency
@@ -163,7 +163,7 @@
         (t/is (sto/object? mobj1))
         (t/is (sto/object? mobj2))
         (t/is (= 122785 (:size mobj1)))
-        (t/is (= 3302 (:size mobj2)))))))
+        (t/is (= 3299 (:size mobj2)))))))
 
 (t/deftest media-object-upload-command
   (let [prof   (th/create-profile* 1)
@@ -200,7 +200,7 @@
         (t/is (sto/object? mobj1))
         (t/is (sto/object? mobj2))
         (t/is (= 312043 (:size mobj1)))
-        (t/is (= 3887   (:size mobj2)))))))
+        (t/is (= 3901   (:size mobj2)))))))
 
 
 (t/deftest media-object-upload-idempotency-command

backend/test/backend_tests/rpc_team_test.clj

@@ -449,6 +449,23 @@
       (t/is (nil? res)))))
 
 
+(t/deftest get-owned-teams
+  (let [profile1 (th/create-profile* 1 {:is-active true})
+        profile2 (th/create-profile* 2 {:is-active true})
+        team1    (th/create-team* 1 {:profile-id (:id profile1)})
+        team2    (th/create-team* 2 {:profile-id (:id profile2)})
+
+        params   {::th/type :get-owned-teams
+                  ::rpc/profile-id (:id profile1)}
+        out      (th/command! params)]
+
+    (t/is (th/success? out))
+    (let [result (:result out)]
+      (t/is (= 1 (count result)))
+      (t/is (= (:id team1) (-> result first :id)))
+      (t/is (not= (:default-team-id profile1) (-> result first :id))))))
+
+
 (t/deftest team-deletion-1
   (let [profile1 (th/create-profile* 1 {:is-active true})
         team     (th/create-team* 1 {:profile-id (:id profile1)})

common/deps.edn

@@ -2,7 +2,7 @@
  {org.clojure/clojure {:mvn/version "1.12.0"}
   org.clojure/data.json {:mvn/version "2.5.1"}
   org.clojure/tools.cli {:mvn/version "1.1.230"}
-  org.clojure/clojurescript {:mvn/version "1.11.132"}
+  org.clojure/clojurescript {:mvn/version "1.12.38"}
   org.clojure/test.check {:mvn/version "1.1.1"}
   org.clojure/data.fressian {:mvn/version "1.1.0"}
 
@@ -12,14 +12,14 @@
   org.apache.logging.log4j/log4j-web {:mvn/version "2.24.3"}
   org.apache.logging.log4j/log4j-jul {:mvn/version "2.24.3"}
   org.apache.logging.log4j/log4j-slf4j2-impl {:mvn/version "2.24.3"}
-  org.slf4j/slf4j-api {:mvn/version "2.0.16"}
+  org.slf4j/slf4j-api {:mvn/version "2.0.17"}
   pl.tkowalcz.tjahzi/log4j2-appender {:mvn/version "0.9.32"}
 
-  selmer/selmer {:mvn/version "1.12.61"}
+  selmer/selmer {:mvn/version "1.12.62"}
   criterium/criterium {:mvn/version "0.4.6"}
 
   metosin/jsonista {:mvn/version "0.3.13"}
-  metosin/malli {:mvn/version "0.17.0"}
+  metosin/malli {:mvn/version "0.18.0"}
 
   expound/expound {:mvn/version "0.9.0"}
   com.cognitect/transit-clj {:mvn/version "1.0.333"}
@@ -30,7 +30,7 @@
   funcool/tubax {:mvn/version "2021.05.20-0"}
   funcool/cuerdas {:mvn/version "2023.11.09-407"}
   funcool/promesa
-  {:git/sha "0c5ed6ad033515a2df4b55addea044f60e9653d0"
+  {:git/sha "f52f58cfacf62f59eab717e2637f37729d0cc383"
    :git/url "https://github.com/funcool/promesa"}
 
   funcool/datoteka
@@ -59,7 +59,7 @@
  {:dev
   {:extra-deps
    {org.clojure/tools.namespace {:mvn/version "RELEASE"}
-    thheller/shadow-cljs {:mvn/version "2.28.20"}
+    thheller/shadow-cljs {:mvn/version "3.0.5"}
     com.clojure-goes-fast/clj-async-profiler {:mvn/version "RELEASE"}
     com.bhauman/rebel-readline {:mvn/version "RELEASE"}
     criterium/criterium {:mvn/version "RELEASE"}
@@ -68,7 +68,7 @@
 
   :build
   {:extra-deps
-   {io.github.clojure/tools.build {:git/tag "v0.10.6" :git/sha "52cf7d6"}}
+   {io.github.clojure/tools.build {:git/tag "v0.10.9" :git/sha "e405aac"}}
    :ns-default build}
 
   :test
@@ -76,9 +76,9 @@
    :extra-deps {lambdaisland/kaocha {:mvn/version "1.91.1392"}}}
 
   :shadow-cljs
-  {:main-opts ["-m" "shadow.cljs.devtools.cli"]}
+  {:main-opts ["-m" "shadow.cljs.devtools.cli"]
+   :jvm-opts ["--sun-misc-unsafe-memory-access=allow"]}
 
   :outdated
   {:extra-deps {com.github.liquidz/antq {:mvn/version "RELEASE"}}
    :main-opts ["-m" "antq.core"]}}}
-

common/package.json

@@ -4,20 +4,19 @@
   "license": "MPL-2.0",
   "author": "Kaleidos INC",
   "private": true,
-  "packageManager": "yarn@4.6.0+sha512.5383cc12567a95f1d668fbe762dfe0075c595b4bfff433be478dbbe24e05251a8e8c3eb992a986667c1d53b6c3a9c85b8398c35a960587fbd9fa3a0915406728",
+  "packageManager": "yarn@4.9.1+sha512.f95ce356460e05be48d66401c1ae64ef84d163dd689964962c6888a9810865e39097a5e9de748876c2e0bf89b232d583c33982773e9903ae7a76257270986538",
   "type": "module",
   "repository": {
     "type": "git",
     "url": "https://github.com/penpot/penpot"
   },
   "dependencies": {
-    "luxon": "^3.4.4",
-    "sax": "^1.4.1"
+    "luxon": "^3.4.4"
   },
   "devDependencies": {
     "concurrently": "^9.0.1",
     "nodemon": "^3.1.7",
-    "shadow-cljs": "2.28.20",
+    "shadow-cljs": "3.0.5",
     "source-map-support": "^0.5.21",
     "ws": "^8.17.0"
   },

common/scripts/repl

@@ -2,16 +2,20 @@
 
 export PENPOT_FLAGS="enable-asserts enable-audit-log $PENPOT_FLAGS"
 
-export OPTIONS="
-       -A:dev \
-       -J-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager \
-       -J-Djdk.attach.allowAttachSelf \
-       -J-Dpolyglot.engine.WarnInterpreterOnly=false \
-       -J-XX:+EnableDynamicAgentLoading \
-       -J-XX:-OmitStackTraceInFastThrow \
-       -J-XX:+UnlockDiagnosticVMOptions \
-       -J-XX:+DebugNonSafepoints \
-       -J-Djdk.tracePinnedThreads=full"
+export JAVA_OPTS="\
+       -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager \
+       -Djdk.attach.allowAttachSelf \
+       -Dlog4j2.configurationFile=log4j2-devenv-repl.xml \
+       -Djdk.tracePinnedThreads=full \
+       -XX:+EnableDynamicAgentLoading \
+       -XX:-OmitStackTraceInFastThrow                    \
+       -XX:+UnlockDiagnosticVMOptions \
+       -XX:+DebugNonSafepoints \
+       --sun-misc-unsafe-memory-access=allow \
+       --enable-preview \
+       --enable-native-access=ALL-UNNAMED";
+
+export OPTIONS="-A:dev"
 
 export OPTIONS_EVAL="nil"
 # export OPTIONS_EVAL="(set! *warn-on-reflection* true)"

common/src/app/common/colors.cljc

@@ -342,13 +342,20 @@
   (-> (hex->hsl data)
       (conj opacity)))
 
-#?(:cljs
-   (defn format-hsla
-     [[h s l a]]
-     (let [precision 2
-           rounded-s (* 100 (parse-double (d/format-precision s precision)))
-           rounded-l (* 100 (parse-double (d/format-precision l precision)))]
-       (str/concat "" h ", " rounded-s "%, " rounded-l "%, " a))))
+(defn format-hsla
+  [[h s l a]]
+  (let [precision 2
+        rounded-h (int h)
+        rounded-s (d/format-number (* 100 s) precision)
+        rounded-l (d/format-number (* 100 l) precision)
+        rounded-a (d/format-number a precision)]
+    (str/concat "" rounded-h ", " rounded-s "%, " rounded-l "%, " rounded-a)))
+
+(defn format-rgba
+  [[r g b a]]
+  (let [precision 2
+        rounded-a (d/format-number a precision)]
+    (str/ffmt "%, %, %, %" r g b rounded-a)))
 
 (defn- hue->rgb
   "Helper for hsl->rgb"

common/src/app/common/data.cljc

@@ -82,15 +82,22 @@
   "Assoc a k v pair, in the order position just before the other key."
   [o ks k v before-k]
   (let [f (fn [o']
-            (cond-> (reduce
-                     (fn [acc [k' v']]
-                       (cond
-                         (and before-k (= k' before-k)) (assoc acc k v k' v')
-                         (= k k') acc
-                         :else (assoc acc k' v')))
-                     (ordered-map)
-                     o')
-              (not before-k) (assoc k v)))]
+            (let [found  (volatile! false)
+                  result (reduce
+                          (fn [acc [k' v']]
+                            (cond
+                              (and before-k (= k' before-k))
+                              (do
+                                (vreset! found true)
+                                (assoc acc k v k' v'))
+
+                              (= k k') acc
+                              :else (assoc acc k' v')))
+                          (ordered-map)
+                          o')]
+              (if (or (not before-k) (not @found))
+                (assoc result k v)
+                result)))]
     (if (seq ks)
       (oupdate-in o ks f)
       (f o))))
@@ -1007,27 +1014,35 @@
 (def ^:const trail-zeros-regex-1 #"\.0+$")
 (def ^:const trail-zeros-regex-2 #"(\.\d*[^0])0+$")
 
-#?(:cljs
-   (defn format-precision
-     "Creates a number with predetermined precision and then removes the trailing 0.
+(defn format-precision
+  "Creates a number with predetermined precision and then removes the trailing 0.
   Examples:
     12.0123, 0 => 12
     12.0123, 1 => 12
     12.0123, 2 => 12.01"
-     [num precision]
+  [num precision]
 
-     (if (number? num)
-       (try
-         (let [num-str (mth/to-fixed num precision)
+  (if (number? num)
+    (try
+      (let [num-str (mth/to-fixed num precision)
                ;; Remove all trailing zeros after the comma 100.00000
-               num-str (str/replace num-str trail-zeros-regex-1 "")]
+            num-str (str/replace num-str trail-zeros-regex-1 "")]
            ;; Remove trailing zeros after a decimal number: 0.001|00|
-           (if-let [m (re-find trail-zeros-regex-2 num-str)]
-             (str/replace num-str (first m) (second m))
-             num-str))
-         (catch :default _
-           (str num)))
-       (str num))))
+        (if-let [m (re-find trail-zeros-regex-2 num-str)]
+          (str/replace num-str (first m) (second m))
+          num-str))
+      (catch #?(:clj Throwable :cljs :default) _
+        (str num)))
+    (str num)))
+
+(defn format-number
+  ([value]
+   (format-number value nil))
+  ([value {:keys [precision] :or {precision 2}}]
+   (let [value (if (string? value) (parse-double value) value)]
+     (when (num? value)
+       (let [value (format-precision value precision)]
+         (str value))))))
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; Util protocols

common/src/app/common/data/macros.cljc

@@ -6,7 +6,7 @@
 
 (ns app.common.data.macros
   "Data retrieval & manipulation specific macros."
-  (:refer-clojure :exclude [get-in select-keys str with-open min max])
+  (:refer-clojure :exclude [get-in select-keys str with-open max])
   #?(:cljs (:require-macros [app.common.data.macros]))
   (:require
    #?(:clj [clojure.core :as c]
@@ -144,3 +144,8 @@
                  (str "expr assert: " (pr-str expr)))]
      (when *assert*
        `(runtime-assert ~hint (fn [] ~expr))))))
+
+(defn truncate
+  "Truncates a string to a certain length"
+  [s max-length]
+  (subs s 0 (min max-length (count s))))

common/src/app/common/data/undo_stack.cljc

@@ -47,7 +47,7 @@
 
 (defn undo
   [stack]
-  (update stack :index dec))
+  (update stack :index #(max 0 (dec %))))
 
 (defn redo
   [{index :index items :items :as stack}]

common/src/app/common/features.cljc

@@ -46,21 +46,30 @@
   #{"fdata/objects-map"
     "fdata/pointer-map"
     "fdata/shape-data-type"
+    "fdata/path-data"
     "components/v2"
     "styles/v2"
     "layout/grid"
     "plugins/runtime"
     "design-tokens/v1"
     "text-editor/v2"
-    "render-wasm/v1"})
+    "render-wasm/v1"
+    "variants/v1"})
 
 ;; A set of features enabled by default
 (def default-features
   #{"fdata/shape-data-type"
+    "fdata/path-data"
     "styles/v2"
     "layout/grid"
     "components/v2"
-    "plugins/runtime"})
+    "plugins/runtime"
+    "design-tokens/v1"})
+
+;; A set of features that should not be propagated to team on creating
+;; or modifying a file
+(def no-team-inheritable-features
+  #{"fdata/path-data"})
 
 ;; A set of features which only affects on frontend and can be enabled
 ;; and disabled freely by the user any time. This features does not
@@ -83,12 +92,12 @@
 ;; be applied (per example backend can operate in both modes with or
 ;; without migration applied)
 (def no-migration-features
-  (-> #{"fdata/objects-map"
-        "fdata/pointer-map"
-        "layout/grid"
+  (-> #{"layout/grid"
+        "design-tokens/v1"
         "fdata/shape-data-type"
-        "design-tokens/v1"}
-      (into frontend-only-features)))
+        "fdata/path-data"}
+      (into frontend-only-features)
+      (into backend-only-features)))
 
 (sm/register!
  ^{::sm/type ::features}
@@ -102,15 +111,14 @@
   "Translate a flag to a feature name"
   [flag]
   (case flag
-    :feature-components-v2 "components/v2"
     :feature-styles-v2 "styles/v2"
-    :feature-grid-layout "layout/grid"
     :feature-fdata-objects-map "fdata/objects-map"
     :feature-fdata-pointer-map "fdata/pointer-map"
     :feature-plugins "plugins/runtime"
     :feature-design-tokens "design-tokens/v1"
     :feature-text-editor-v2 "text-editor/v2"
     :feature-render-wasm "render-wasm/v1"
+    :feature-variants "variants/v1"
     nil))
 
 (defn migrate-legacy-features
@@ -155,7 +163,6 @@
         team-features    (into #{} xf-remove-ephimeral (:features team))]
     (-> enabled-features
         (set/intersection no-migration-features)
-        (set/difference frontend-only-features)
         (set/union team-features))))
 
 (defn check-client-features!
@@ -164,6 +171,8 @@
   frontend client"
   [enabled-features client-features]
   (when (set? client-features)
+    ;; Check if client declares support for features enabled on
+    ;; backend side
     (let [not-supported (-> enabled-features
                             (set/difference client-features)
                             (set/difference frontend-only-features)
@@ -173,14 +182,6 @@
                   :code :feature-not-supported
                   :feature (first not-supported)
                   :hint (str/ffmt "client declares no support for '%' features"
-                                  (str/join "," not-supported)))))
-
-    (let [not-supported (set/difference client-features supported-features)]
-      (when (seq not-supported)
-        (ex/raise :type :restriction
-                  :code :feature-not-supported
-                  :feature (first not-supported)
-                  :hint (str/ffmt "backend does not support '%' features requested by client"
                                   (str/join "," not-supported))))))
 
   enabled-features)
@@ -191,57 +192,55 @@
   supported by the current backend"
   [enabled-features]
   (let [not-supported (set/difference enabled-features supported-features)]
-    (when (seq not-supported)
+    (when-let [not-supported (first not-supported)]
       (ex/raise :type :restriction
                 :code :feature-not-supported
-                :feature (first not-supported)
-                :hint (str/ffmt "features '%' not supported"
-                                (str/join "," not-supported)))))
-  enabled-features)
+                :feature not-supported
+                :hint (str/ffmt "feature '%' not supported on this backend" not-supported)))
+    enabled-features))
 
 (defn check-file-features!
   "Function used for check feature compability between currently
   enabled features set on backend with the provided featured set by
   the penpot file"
-  ([enabled-features file-features]
-   (check-file-features! enabled-features file-features #{}))
-  ([enabled-features file-features client-features]
-   (let [file-features   (into #{} xf-remove-ephimeral file-features)
-         ;; We should ignore all features that does not match with the
-         ;; `no-migration-features` set because we can't enable them
-         ;; as-is, because they probably need migrations
-         client-features (set/intersection client-features no-migration-features)]
-     (let [not-supported (-> enabled-features
-                             (set/union client-features)
-                             (set/difference file-features)
-                             ;; NOTE: we don't want to raise a feature-mismatch
-                             ;; exception for features which don't require an
-                             ;; explicit file migration process or has no real
-                             ;; effect on file data structure
-                             (set/difference no-migration-features))]
-       (when (seq not-supported)
-         (ex/raise :type :restriction
-                   :code :file-feature-mismatch
-                   :feature (first not-supported)
-                   :hint (str/ffmt "enabled features '%' not present in file (missing migration)"
-                                   (str/join "," not-supported)))))
+  [enabled-features file-features]
+  (let [file-features (into #{} xf-remove-ephimeral file-features)
+        not-supported (-> enabled-features
+                          (set/difference file-features)
+                          ;; NOTE: we don't want to raise a feature-mismatch
+                          ;; exception for features which don't require an
+                          ;; explicit file migration process or has no real
+                          ;; effect on file data structure
+                          (set/difference no-migration-features))]
 
-     (check-supported-features! file-features)
+    (when-let [not-supported (first not-supported)]
+      (ex/raise :type :restriction
+                :code :file-feature-mismatch
+                :feature not-supported
+                :hint (str/ffmt "enabled feature '%' not present in file (missing migration)"
+                                not-supported)))
 
-     (let [not-supported (-> file-features
-                             (set/difference enabled-features)
-                             (set/difference client-features)
-                             (set/difference backend-only-features)
-                             (set/difference frontend-only-features))]
+    (check-supported-features! file-features)
 
-       (when (seq not-supported)
-         (ex/raise :type :restriction
-                   :code :file-feature-mismatch
-                   :feature (first not-supported)
-                   :hint (str/ffmt "file features '%' not enabled"
-                                   (str/join "," not-supported))))))
+    ;; Components v1 is deprecated
+    (when-not (contains? file-features "components/v2")
+      (ex/raise :type :restriction
+                :code :file-in-components-v1
+                :hint "components v1 is deprecated"))
 
-   enabled-features))
+    (let [not-supported (-> file-features
+                            (set/difference enabled-features)
+                            (set/difference backend-only-features)
+                            (set/difference frontend-only-features))]
+
+      ;; Check if file has a feature but that feature is not enabled
+      (when-let [not-supported (first not-supported)]
+        (ex/raise :type :restriction
+                  :code :file-feature-mismatch
+                  :feature not-supported
+                  :hint (str/ffmt "file feature '%' not enabled" not-supported))))
+
+    enabled-features))
 
 (defn check-teams-compatibility!
   [{source-features :features} {destination-features :features}]

common/src/app/common/files/changes.cljc

@@ -26,11 +26,10 @@
    [app.common.types.pages-list :as ctpl]
    [app.common.types.shape :as cts]
    [app.common.types.shape-tree :as ctst]
-   [app.common.types.token :as cto]
-   [app.common.types.token-theme :as ctot]
    [app.common.types.tokens-lib :as ctob]
    [app.common.types.typographies-list :as ctyl]
    [app.common.types.typography :as ctt]
+   [app.common.types.variant :as ctv]
    [app.common.uuid :as uuid]
    [clojure.set :as set]))
 
@@ -48,14 +47,14 @@
      [:type [:= :assign]]
      ;; NOTE: the full decoding is happening on the handler because it
      ;; needs a proper context of the current shape and its type
-     [:value [:map-of :keyword :any]]
+     [:value [:map-of :keyword ::sm/any]]
      [:ignore-touched {:optional true} :boolean]
      [:ignore-geometry {:optional true} :boolean]]]
    [:set
     [:map {:title "SetOperation"}
      [:type [:= :set]]
      [:attr :keyword]
-     [:val :any]
+     [:val ::sm/any]
      [:ignore-touched {:optional true} :boolean]
      [:ignore-geometry {:optional true} :boolean]]]
    [:set-touched
@@ -239,9 +238,9 @@
       [:component-id {:optional true} ::sm/uuid]
       [:ignore-touched {:optional true} :boolean]
       [:parent-id ::sm/uuid]
-      [:shapes :any]
+      [:shapes ::sm/any]
       [:index {:optional true} [:maybe :int]]
-      [:after-shape {:optional true} :any]
+      [:after-shape {:optional true} ::sm/any]
       [:component-swap {:optional true} :boolean]]]
 
     [:reorder-children
@@ -251,14 +250,14 @@
       [:component-id {:optional true} ::sm/uuid]
       [:ignore-touched {:optional true} :boolean]
       [:parent-id ::sm/uuid]
-      [:shapes :any]]]
+      [:shapes ::sm/any]]]
 
     [:add-page
      [:map {:title "AddPageChange"}
       [:type [:= :add-page]]
       [:id {:optional true} ::sm/uuid]
       [:name {:optional true} :string]
-      [:page {:optional true} :any]]]
+      [:page {:optional true} ::sm/any]]]
 
     [:mod-page
      [:map {:title "ModPageChange"}
@@ -311,12 +310,12 @@
     [:add-media
      [:map {:title "AddMediaChange"}
       [:type [:= :add-media]]
-      [:object ::ctf/media-object]]]
+      [:object ctf/schema:media]]]
 
     [:mod-media
      [:map {:title "ModMediaChange"}
       [:type [:= :mod-media]]
-      [:object ::ctf/media-object]]]
+      [:object ctf/schema:media]]]
 
     [:del-media
      [:map {:title "DelMediaChange"}
@@ -328,21 +327,25 @@
       [:type [:= :add-component]]
       [:id ::sm/uuid]
       [:name :string]
-      [:shapes {:optional true} [:vector {:gen/max 3} :any]]
+      [:shapes {:optional true} [:vector {:gen/max 3} ::sm/any]]
       [:path {:optional true} :string]]]
 
     [:mod-component
      [:map {:title "ModCompoenentChange"}
       [:type [:= :mod-component]]
       [:id ::sm/uuid]
-      [:shapes {:optional true} [:vector {:gen/max 3} :any]]
-      [:name {:optional true} :string]]]
+      [:shapes {:optional true} [:vector {:gen/max 3} ::sm/any]]
+      [:name {:optional true} :string]
+      [:variant-id {:optional true} ::sm/uuid]
+      [:variant-properties {:optional true} [:vector ::ctv/variant-property]]]]
 
     [:del-component
      [:map {:title "DelComponentChange"}
       [:type [:= :del-component]]
       [:id ::sm/uuid]
-      [:main-instance {:optional true} :any]
+      ;; when it is an undo of a cut-paste, we need to undo the movement
+      ;; of the shapes so we need to move them delta
+      [:delta {:optional true} ::gpt/point]
       [:skip-undelete? {:optional true} :boolean]]]
 
     [:restore-component
@@ -371,111 +374,63 @@
       [:type [:= :del-typography]]
       [:id ::sm/uuid]]]
 
-    [:add-temporary-token-theme
-     [:map {:title "AddTemporaryTokenThemeChange"}
-      [:type [:= :add-temporary-token-theme]]
-      [:token-theme ::ctot/token-theme]]]
-
     [:update-active-token-themes
      [:map {:title "UpdateActiveTokenThemes"}
       [:type [:= :update-active-token-themes]]
       [:theme-ids [:set :string]]]]
 
-    [:delete-temporary-token-theme
-     [:map {:title "DeleteTemporaryTokenThemeChange"}
-      [:type [:= :delete-temporary-token-theme]]
-      [:id ::sm/uuid]
-      [:name :string]]]
-
-    [:add-token-theme
-     [:map {:title "AddTokenThemeChange"}
-      [:type [:= :add-token-theme]]
-      [:token-theme ::ctot/token-theme]]]
-
-    [:mod-token-theme
-     [:map {:title "ModTokenThemeChange"}
-      [:type [:= :mod-token-theme]]
-      [:group :string]
-      [:name :string]
-      [:token-theme ::ctot/token-theme]]]
-
-    [:del-token-theme
-     [:map {:title "DelTokenThemeChange"}
-      [:type [:= :del-token-theme]]
-      [:group :string]
-      [:name :string]]]
-
-    [:add-token-set
-     [:map {:title "AddTokenSetChange"}
-      [:type [:= :add-token-set]]
-      [:token-set ::ctot/token-set]]]
-
-    [:add-token-sets
-     [:map {:title "AddTokenSetsChange"}
-      [:type [:= :add-token-sets]]
-      [:token-sets [:sequential ::ctot/token-set]]]]
-
     [:rename-token-set-group
      [:map {:title "RenameTokenSetGroup"}
       [:type [:= :rename-token-set-group]]
       [:set-group-path [:vector :string]]
       [:set-group-fname :string]]]
 
-    [:mod-token-set
-     [:map {:title "ModTokenSetChange"}
-      [:type [:= :mod-token-set]]
-      [:name :string]
-      [:token-set ::ctot/token-set]]]
-
-    [:move-token-set-before
-     [:map {:title "MoveTokenSetBefore"}
-      [:type [:= :move-token-set-before]]
+    [:move-token-set
+     [:map {:title "MoveTokenSet"}
+      [:type [:= :move-token-set]]
       [:from-path [:vector :string]]
       [:to-path [:vector :string]]
       [:before-path [:maybe [:vector :string]]]
-      [:before-group? [:maybe :boolean]]]]
+      [:before-group [:maybe :boolean]]]]
 
-    [:move-token-set-group-before
-     [:map {:title "MoveTokenSetGroupBefore"}
-      [:type [:= :move-token-set-group-before]]
+    [:move-token-set-group
+     [:map {:title "MoveTokenSetGroup"}
+      [:type [:= :move-token-set-group]]
       [:from-path [:vector :string]]
       [:to-path [:vector :string]]
       [:before-path [:maybe [:vector :string]]]
-      [:before-group? [:maybe :boolean]]]]
+      [:before-group [:maybe :boolean]]]]
 
-    [:del-token-set
-     [:map {:title "DelTokenSetChange"}
-      [:type [:= :del-token-set]]
-      [:name :string]]]
-
-    [:del-token-set-path
-     [:map {:title "DelTokenSetPathChange"}
-      [:type [:= :del-token-set-path]]
-      [:path :string]]]
+    [:set-token-theme
+     [:map {:title "SetTokenThemeChange"}
+      [:type [:= :set-token-theme]]
+      [:theme-name :string]
+      [:group :string]
+      [:theme [:maybe ctob/schema:token-theme-attrs]]]]
 
     [:set-tokens-lib
      [:map {:title "SetTokensLib"}
       [:type [:= :set-tokens-lib]]
-      [:tokens-lib :any]]]
+      [:tokens-lib ::sm/any]]]
 
-    [:add-token
-     [:map {:title "AddTokenChange"}
-      [:type [:= :add-token]]
+    [:set-token-set
+     [:map {:title "SetTokenSetChange"}
+      [:type [:= :set-token-set]]
       [:set-name :string]
-      [:token ::cto/token]]]
+      [:group? :boolean]
+      [:token-set [:maybe ctob/schema:token-set-attrs]]]]
 
-    [:mod-token
-     [:map {:title "ModTokenChange"}
-      [:type [:= :mod-token]]
+    [:set-token
+     [:map {:title "SetTokenChange"}
+      [:type [:= :set-token]]
       [:set-name :string]
-      [:name :string]
-      [:token ::cto/token]]]
+      [:token-name :string]
+      [:token [:maybe ctob/schema:token-attrs]]]]
 
-    [:del-token
-     [:map {:title "DelTokenChange"}
-      [:type [:= :del-token]]
-      [:set-name :string]
-      [:name :string]]]]])
+    [:set-base-font-size
+     [:map {:title "ModBaseFontSize"}
+      [:type [:= :set-base-font-size]]
+      [:base-font-size :string]]]]])
 
 (def schema:changes
   [:sequential {:gen/max 5 :gen/min 1} schema:change])
@@ -782,20 +737,22 @@
 
           (update-group [group objects]
             (let [lookup   (d/getf objects)
-                  children (->> group :shapes (map lookup))]
+                  children (get group :shapes)]
               (cond
                 ;; If the group is empty we don't make any changes. Will be removed by a later process
                 (empty? children)
                 group
 
                 (= :bool (:type group))
-                (gsh/update-bool-selrect group children objects)
+                (gsh/update-bool group objects)
 
                 (:masked-group group)
-                (set-mask-selrect group children)
+                (->> (map lookup children)
+                     (set-mask-selrect group))
 
                 :else
-                (gsh/update-group-selrect group children))))]
+                (->> (map lookup children)
+                     (gsh/update-group-selrect group)))))]
 
     (if page-id
       (d/update-in-when data [:pages-index page-id :objects] reg-objects)
@@ -1009,8 +966,8 @@
   (ctkl/mod-component data params))
 
 (defmethod process-change :del-component
-  [data {:keys [id skip-undelete? main-instance]}]
-  (ctf/delete-component data id skip-undelete? main-instance))
+  [data {:keys [id skip-undelete? delta]}]
+  (ctf/delete-component data id skip-undelete? delta))
 
 (defmethod process-change :restore-component
   [data {:keys [id page-id]}]
@@ -1040,80 +997,63 @@
   [data {:keys [tokens-lib]}]
   (assoc data :tokens-lib tokens-lib))
 
-(defmethod process-change :add-token
-  [data {:keys [set-name token]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/add-token-in-set set-name (ctob/make-token token)))))
+(defmethod process-change :set-token
+  [data {:keys [set-name token-name token]}]
+  (update data :tokens-lib
+          (fn [lib]
+            (let [lib' (ctob/ensure-tokens-lib lib)]
+              (cond
+                (not token)
+                (ctob/delete-token-from-set lib' set-name token-name)
 
-(defmethod process-change :mod-token
-  [data {:keys [set-name name token]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/update-token-in-set
-                                 set-name
-                                 name
-                                 (fn [old-token]
-                                   (ctob/make-token (merge old-token token)))))))
+                (not (ctob/get-token-in-set lib' set-name token-name))
+                (ctob/add-token-in-set lib' set-name (ctob/make-token token))
 
-(defmethod process-change :del-token
-  [data {:keys [set-name name]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/delete-token-from-set
-                                 set-name
-                                 name))))
+                :else
+                (ctob/update-token-in-set lib' set-name token-name (fn [prev-token]
+                                                                     (ctob/make-token (merge prev-token token)))))))))
 
-(defmethod process-change :add-temporary-token-theme
-  [data {:keys [token-theme]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/add-theme (ctob/make-token-theme token-theme)))))
+(defmethod process-change :set-token-set
+  [data {:keys [set-name group? token-set]}]
+  (update data :tokens-lib
+          (fn [lib]
+            (let [lib' (ctob/ensure-tokens-lib lib)]
+              (cond
+                (not token-set)
+                (if group?
+                  (ctob/delete-set-group lib' set-name)
+                  (ctob/delete-set lib' set-name))
+
+                (not (ctob/get-set lib' set-name))
+                (ctob/add-set lib' (ctob/make-token-set token-set))
+
+                :else
+                (ctob/update-set lib' set-name (fn [prev-token-set]
+                                                 (ctob/make-token-set (merge prev-token-set token-set)))))))))
+
+(defmethod process-change :set-token-theme
+  [data {:keys [group theme-name theme]}]
+  (update data :tokens-lib
+          (fn [lib]
+            (let [lib' (ctob/ensure-tokens-lib lib)]
+              (cond
+                (not theme)
+                (ctob/delete-theme lib' group theme-name)
+
+                (not (ctob/get-theme lib' group theme-name))
+                (ctob/add-theme lib' (ctob/make-token-theme theme))
+
+                :else
+                (ctob/update-theme lib'
+                                   group theme-name
+                                   (fn [prev-token-theme]
+                                     (ctob/make-token-theme (merge prev-token-theme theme)))))))))
 
 (defmethod process-change :update-active-token-themes
   [data {:keys [theme-ids]}]
   (update data :tokens-lib #(-> % (ctob/ensure-tokens-lib)
                                 (ctob/set-active-themes theme-ids))))
 
-(defmethod process-change :delete-temporary-token-theme
-  [data {:keys [group name]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/delete-theme group name))))
-
-(defmethod process-change :add-token-theme
-  [data {:keys [token-theme]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/add-theme (-> token-theme
-                                                    (ctob/make-token-theme))))))
-
-(defmethod process-change :mod-token-theme
-  [data {:keys [name group token-theme]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/update-theme group name
-                                                   (fn [prev-theme]
-                                                     (merge prev-theme token-theme))))))
-
-(defmethod process-change :del-token-theme
-  [data {:keys [group name]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/delete-theme group name))))
-
-(defmethod process-change :add-token-set
-  [data {:keys [token-set]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/add-set (ctob/make-token-set token-set)))))
-
-(defmethod process-change :add-token-sets
-  [data {:keys [token-sets]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/add-sets (map ctob/make-token-set token-sets)))))
-
 (defmethod process-change :rename-token-set-group
   [data {:keys [set-group-path set-group-fname]}]
   (update data :tokens-lib (fn [lib]
@@ -1121,37 +1061,24 @@
                                  (ctob/ensure-tokens-lib)
                                  (ctob/rename-set-group set-group-path set-group-fname)))))
 
-(defmethod process-change :mod-token-set
-  [data {:keys [name token-set]}]
-  (update data :tokens-lib (fn [lib]
-                             (-> lib
-                                 (ctob/ensure-tokens-lib)
-                                 (ctob/update-set name (fn [prev-set]
-                                                         (merge prev-set (dissoc token-set :tokens))))))))
-
-(defmethod process-change :move-token-set-before
-  [data {:keys [from-path to-path before-path before-group?] :as changes}]
+(defmethod process-change :move-token-set
+  [data {:keys [from-path to-path before-path before-group] :as changes}]
   (update data :tokens-lib #(-> %
                                 (ctob/ensure-tokens-lib)
-                                (ctob/move-set from-path to-path before-path before-group?))))
+                                (ctob/move-set from-path to-path before-path before-group))))
 
-(defmethod process-change :move-token-set-group-before
-  [data {:keys [from-path to-path before-path before-group?]}]
+(defmethod process-change :move-token-set-group
+  [data {:keys [from-path to-path before-path before-group]}]
   (update data :tokens-lib #(-> %
                                 (ctob/ensure-tokens-lib)
-                                (ctob/move-set-group from-path to-path before-path before-group?))))
+                                (ctob/move-set-group from-path to-path before-path before-group))))
 
-(defmethod process-change :del-token-set
-  [data {:keys [name]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/delete-set-path name))))
+;; === Base font size
+
+(defmethod process-change :set-base-font-size
+  [data {:keys [base-font-size]}]
+  (ctf/set-base-font-size data base-font-size))
 
-(defmethod process-change :del-token-set-path
-  [data {:keys [path]}]
-  (update data :tokens-lib #(-> %
-                                (ctob/ensure-tokens-lib)
-                                (ctob/delete-set-path path))))
 
 ;; === Operations
 

common/src/app/common/files/changes_builder.cljc

@@ -8,7 +8,6 @@
   (:require
    [app.common.data :as d]
    [app.common.data.macros :as dm]
-   [app.common.features :as cfeat]
    [app.common.files.changes :as cfc]
    [app.common.files.helpers :as cfh]
    [app.common.geom.matrix :as gmt]
@@ -25,18 +24,19 @@
 
 ;; Auxiliary functions to help create a set of changes (undo + redo)
 
-(sm/register!
- ^{::sm/type ::changes}
- [:map {:title "changes"}
-  [:redo-changes vector?]
-  [:undo-changes seq?]
-  [:origin {:optional true} any?]
-  [:save-undo? {:optional true} boolean?]
-  [:stack-undo? {:optional true} boolean?]
-  [:undo-group {:optional true} any?]])
+(def schema:changes
+  (sm/register!
+   ^{::sm/type ::changes}
+   [:map {:title "changes"}
+    [:redo-changes vector?]
+    [:undo-changes seq?]
+    [:origin {:optional true} ::sm/any]
+    [:save-undo? {:optional true} boolean?]
+    [:stack-undo? {:optional true} boolean?]
+    [:undo-group {:optional true} ::sm/any]]))
 
 (def check-changes!
-  (sm/check-fn ::changes))
+  (sm/check-fn schema:changes))
 
 (defn empty-changes
   ([origin page-id]
@@ -84,8 +84,7 @@
 
 (defn with-objects
   [changes objects]
-  (let [fdata (binding [cfeat/*current* #{"components/v2"}]
-                (ctf/make-file-data (uuid/next) uuid/zero))
+  (let [fdata (ctf/make-file-data (uuid/next) uuid/zero)
         fdata (assoc-in fdata [:pages-index uuid/zero :objects] objects)]
     (vary-meta changes assoc
                ::file-data fdata
@@ -126,28 +125,41 @@
 ; TODO: remove this when not needed
 (defn- assert-page-id!
   [changes]
-  (dm/assert!
-   "Give a page-id or call (with-page) before using this function"
-   (contains? (meta changes) ::page-id)))
+  (assert
+   (contains? (meta changes) ::page-id)
+   "Give a page-id or call (with-page) before using this function"))
+
+(defn- assert-page!
+  [changes]
+  (assert
+   (contains? (meta changes) ::page)
+   "Give a page or call (with-page) before using this function"))
 
 (defn- assert-container-id!
   [changes]
-  (dm/assert!
-   "Give a page-id or call (with-container) before using this function"
+  (assert
    (or (contains? (meta changes) ::page-id)
-       (contains? (meta changes) ::component-id))))
+       (contains? (meta changes) ::component-id))
+   "Give a page-id or call (with-container) before using this function"))
 
 (defn- assert-objects!
   [changes]
-  (dm/assert!
-   "Call (with-objects) before using this function"
-   (contains? (meta changes) ::file-data)))
+  (assert
+   (contains? (meta changes) ::file-data)
+   "Call (with-objects) before using this function"))
 
 (defn- assert-library!
   [changes]
-  (dm/assert!
-   "Call (with-library-data) before using this function"
-   (contains? (meta changes) ::library-data)))
+  (assert
+   (contains? (meta changes) ::library-data)
+   "Call (with-library-data) before using this function"))
+
+(defn- assert-file-data!
+  [changes]
+  (assert
+   (contains? (meta changes) ::file-data)
+   "Call (with-file-data) before using this function"))
+
 
 (defn- lookup-objects
   [changes]
@@ -155,13 +167,14 @@
     (dm/get-in data [:pages-index uuid/zero :objects])))
 
 (defn apply-changes-local
-  [changes]
-  (dm/assert!
-   "expected valid changes"
-   (check-changes! changes))
+  [changes & {:keys [apply-to-library?]}]
+  (assert
+   (check-changes! changes)
+   "expected valid changes")
 
   (if-let [file-data (::file-data (meta changes))]
-    (let [index         (::applied-changes-count (meta changes))
+    (let [library-data  (::library-data (meta changes))
+          index         (::applied-changes-count (meta changes))
           redo-changes  (:redo-changes changes)
           new-changes   (if (< index (count redo-changes))
                           (->> (subvec (:redo-changes changes) index)
@@ -169,8 +182,12 @@
                                          (assoc :page-id uuid/zero)
                                          (dissoc :component-id))))
                           [])
-          new-file-data (cfc/process-changes file-data new-changes)]
+          new-file-data (cfc/process-changes file-data new-changes)
+          new-library-data (if apply-to-library?
+                             (cfc/process-changes library-data new-changes)
+                             library-data)]
       (vary-meta changes assoc ::file-data new-file-data
+                 ::library-data new-library-data
                  ::applied-changes-count (count redo-changes)))
     changes))
 
@@ -192,6 +209,7 @@
 
 (defn mod-page
   ([changes options]
+   (assert-page! changes)
    (let [page (::page (meta changes))]
      (mod-page changes page options)))
 
@@ -222,6 +240,7 @@
   ([changes type id namespace key value]
    (set-plugin-data changes type id nil namespace key value))
   ([changes type id page-id namespace key value]
+   (assert-file-data! changes)
    (let [data (::file-data (meta changes))
          old-val
          (case type
@@ -288,6 +307,8 @@
 
 (defn set-guide
   [changes id guide]
+  (assert-page-id! changes)
+  (assert-page! changes)
   (let [page-id (::page-id (meta changes))
         page    (::page (meta changes))
         old-val (dm/get-in page [:guides id])]
@@ -301,8 +322,11 @@
                                     :page-id page-id
                                     :id id
                                     :params old-val}))))
+
 (defn set-flow
   [changes id flow]
+  (assert-page-id! changes)
+  (assert-page! changes)
   (let [page-id (::page-id (meta changes))
         page    (::page (meta changes))
         old-val (dm/get-in page [:flows id])
@@ -321,6 +345,8 @@
 
 (defn set-comment-thread-position
   [changes {:keys [id frame-id position] :as thread}]
+  (assert-page-id! changes)
+  (assert-page! changes)
   (let [page-id (::page-id (meta changes))
         page    (::page (meta changes))
 
@@ -342,6 +368,8 @@
 
 (defn set-default-grid
   [changes type params]
+  (assert-page-id! changes)
+  (assert-page! changes)
   (let [page-id (::page-id (meta changes))
         page    (::page (meta changes))
         old-val (dm/get-in page [:grids type])
@@ -475,9 +503,12 @@
           (let [old-val (get old attr)
                 new-val (get new attr)]
             (not= old-val new-val)))
-        new-obj (if with-objects?
-                  (update-fn object objects)
-                  (update-fn object))]
+
+        new-obj
+        (if with-objects?
+          (update-fn object objects)
+          (update-fn object))]
+
     (when-not (= object new-obj)
       (let [attrs (or attrs (d/concat-set (keys object) (keys new-obj)))]
         (filter (partial changed? object new-obj) attrs)))))
@@ -492,6 +523,7 @@
                            :or {ignore-geometry? false ignore-touched false with-objects? false}}]
    (assert-container-id! changes)
    (assert-objects! changes)
+   (assert-page-id! changes)
    (let [page-id      (::page-id (meta changes))
          component-id (::component-id (meta changes))
          objects      (lookup-objects changes)
@@ -654,9 +686,13 @@
                                  nil               ;; so it does not need resize
 
                                  (= (:type parent) :bool)
-                                 (gsh/update-bool-selrect parent children objects)
+                                 (gsh/update-bool parent objects)
 
                                  (= (:type parent) :group)
+                                 ;; FIXME: this functions should be
+                                 ;; normalized in the same way as
+                                 ;; update-bool in order to make all
+                                 ;; this code consistent
                                  (if (:masked-group parent)
                                    (gsh/update-mask-selrect parent children)
                                    (gsh/update-group-selrect parent children)))]
@@ -762,13 +798,6 @@
         (update :undo-changes conj {:type :add-typography :typography prev-typography})
         (apply-changes-local))))
 
-(defn add-temporary-token-theme
-  [changes token-theme]
-  (-> changes
-      (update :redo-changes conj {:type :add-temporary-token-theme :token-theme token-theme})
-      (update :undo-changes conj {:type :delete-temporary-token-theme :id (:id token-theme) :name (:name token-theme)})
-      (apply-changes-local)))
-
 (defn update-active-token-themes
   [changes token-active-theme-ids prev-token-active-theme-ids]
   (-> changes
@@ -776,42 +805,32 @@
       (update :undo-changes conj {:type :update-active-token-themes :theme-ids prev-token-active-theme-ids})
       (apply-changes-local)))
 
-(defn add-token-theme
-  [changes token-theme]
-  (-> changes
-      (update :redo-changes conj {:type :add-token-theme :token-theme token-theme})
-      (update :undo-changes conj {:type :del-token-theme :group (:group token-theme) :name (:name token-theme)})
-      (apply-changes-local)))
-
-(defn update-token-theme
-  [changes token-theme prev-token-theme]
-  (let [name (or (:name prev-token-theme)
-                 (:name token-theme))
-        group (or (:group prev-token-theme)
-                  (:group token-theme))]
-    (-> changes
-        (update :redo-changes conj {:type :mod-token-theme :group group :name name :token-theme token-theme})
-        (update :undo-changes conj {:type :mod-token-theme :group group :name name :token-theme (or prev-token-theme token-theme)})
-        (apply-changes-local))))
-
-(defn delete-token-theme
-  [changes group name]
+(defn set-token-theme [changes group theme-name theme]
   (assert-library! changes)
   (let [library-data (::library-data (meta changes))
-        prev-token-theme (some-> (get library-data :tokens-lib)
-                                 (ctob/get-theme group name))]
+        prev-theme (some-> (get library-data :tokens-lib)
+                           (ctob/get-theme group theme-name))]
     (-> changes
-        (update :redo-changes conj {:type :del-token-theme :group group :name name})
-        (update :undo-changes conj {:type :add-token-theme :token-theme prev-token-theme})
+        (update :redo-changes conj {:type :set-token-theme
+                                    :theme-name theme-name
+                                    :group group
+                                    :theme theme})
+        (update :undo-changes conj (if prev-theme
+                                     {:type :set-token-theme
+                                      :group group
+                                      :theme-name (or
+                                                   ;; Undo of edit
+                                                   (:name theme)
+                                                   ;; Undo of delete
+                                                   theme-name)
+                                      :theme prev-theme}
+                                     ;; Undo of create
+                                     {:type :set-token-theme
+                                      :group group
+                                      :theme-name theme-name
+                                      :theme nil}))
         (apply-changes-local))))
 
-(defn add-token-set
-  [changes token-set]
-  (-> changes
-      (update :redo-changes conj {:type :add-token-set :token-set token-set})
-      (update :undo-changes conj {:type :del-token-set :name (:name token-set)})
-      (apply-changes-local)))
-
 (defn rename-token-set-group
   [changes set-group-path set-group-fname]
   (let [undo-path (ctob/replace-last-path-name set-group-path set-group-fname)
@@ -821,62 +840,39 @@
         (update :undo-changes conj {:type :rename-token-set-group :set-group-path undo-path :set-group-fname undo-fname})
         (apply-changes-local))))
 
-(defn update-token-set
-  [changes token-set prev-token-set]
-  (-> changes
-      (update :redo-changes conj {:type :mod-token-set :name (:name prev-token-set) :token-set token-set})
-      (update :undo-changes conj {:type :mod-token-set :name (:name token-set) :token-set (or prev-token-set token-set)})
-      (apply-changes-local)))
-
-(defn delete-token-set-path
-  [changes group? path]
-  (assert-library! changes)
-  (let [;; TODO Move leaking prefix to library
-        prefixed-path (if group?
-                        (ctob/set-group-path->set-group-prefixed-path path)
-                        (ctob/set-full-path->set-prefixed-full-path path))
-        prefixed-path-str (ctob/join-set-path prefixed-path)
-        library-data (::library-data (meta changes))
-        prev-token-sets (some-> (get library-data :tokens-lib)
-                                (ctob/get-path-sets prefixed-path-str))]
-    (-> changes
-        (update :redo-changes conj {:type :del-token-set-path :path prefixed-path-str})
-        (update :undo-changes conj {:type :add-token-sets :token-sets prev-token-sets})
-        (apply-changes-local))))
-
-(defn move-token-set-before
+(defn move-token-set
   [changes {:keys [from-path to-path before-path before-group? prev-before-path prev-before-group?] :as opts}]
   (-> changes
-      (update :redo-changes conj {:type :move-token-set-before
+      (update :redo-changes conj {:type :move-token-set
                                   :from-path from-path
                                   :to-path to-path
                                   :before-path before-path
-                                  :before-group? before-group?})
-      (update :undo-changes conj {:type :move-token-set-before
+                                  :before-group before-group?})
+      (update :undo-changes conj {:type :move-token-set
                                   :from-path to-path
                                   :to-path from-path
                                   :before-path prev-before-path
-                                  :before-group? prev-before-group?})
+                                  :before-group prev-before-group?})
       (apply-changes-local)))
 
-(defn move-token-set-group-before
+(defn move-token-set-group
   [changes {:keys [from-path to-path before-path before-group? prev-before-path prev-before-group?]}]
-  (prn prev-before-path prev-before-group?)
   (-> changes
-      (update :redo-changes conj {:type :move-token-set-group-before
+      (update :redo-changes conj {:type :move-token-set-group
                                   :from-path from-path
                                   :to-path to-path
                                   :before-path before-path
-                                  :before-group? before-group?})
-      (update :undo-changes conj {:type :move-token-set-group-before
+                                  :before-group before-group?})
+      (update :undo-changes conj {:type :move-token-set-group
                                   :from-path to-path
                                   :to-path from-path
                                   :before-path prev-before-path
-                                  :before-group? prev-before-group?})
+                                  :before-group prev-before-group?})
       (apply-changes-local)))
 
 (defn set-tokens-lib
   [changes tokens-lib]
+  (assert-library! changes)
   (let [library-data (::library-data (meta changes))
         prev-tokens-lib (get library-data :tokens-lib)]
     (-> changes
@@ -884,36 +880,84 @@
         (update :undo-changes conj {:type :set-tokens-lib :tokens-lib prev-tokens-lib})
         (apply-changes-local))))
 
-(defn add-token
-  [changes set-name token]
-  (-> changes
-      (update :redo-changes conj {:type :add-token :set-name set-name :token token})
-      (update :undo-changes conj {:type :del-token :set-name set-name :name (:name token)})
-      (apply-changes-local)))
-
-(defn update-token
-  [changes set-name token prev-token]
-  (-> changes
-      (update :redo-changes conj {:type :mod-token :set-name set-name :name (:name prev-token) :token token})
-      (update :undo-changes conj {:type :mod-token :set-name set-name :name (:name token) :token (or prev-token token)})
-      (apply-changes-local)))
-
-(defn delete-token
-  [changes set-name token-name]
+(defn set-token [changes set-name token-name token]
   (assert-library! changes)
   (let [library-data (::library-data (meta changes))
         prev-token (some-> (get library-data :tokens-lib)
                            (ctob/get-set set-name)
                            (ctob/get-token token-name))]
     (-> changes
-        (update :redo-changes conj {:type :del-token :set-name set-name :name token-name})
-        (update :undo-changes conj {:type :add-token :set-name set-name :token prev-token})
+        (update :redo-changes conj {:type :set-token
+                                    :set-name set-name
+                                    :token-name token-name
+                                    :token token})
+        (update :undo-changes conj (if prev-token
+                                     {:type :set-token
+                                      :set-name set-name
+                                      :token-name (or
+                                                   ;; Undo of edit
+                                                   (:name token)
+                                                   ;; Undo of delete
+                                                   token-name)
+                                      :token prev-token}
+                                     ;; Undo of create token
+                                     {:type :set-token
+                                      :set-name set-name
+                                      :token-name token-name
+                                      :token nil}))
+        (apply-changes-local))))
+
+(defn rename-token-set
+  [changes name new-name]
+
+  (assert-library! changes)
+  (let [library-data   (::library-data (meta changes))
+        prev-token-set (some-> (get library-data :tokens-lib)
+                               (ctob/get-set name))]
+    (-> changes
+        (update :redo-changes conj {:type :set-token-set
+                                    :set-name name
+                                    :token-set (assoc prev-token-set :name new-name)
+                                    :group? false})
+        (update :undo-changes conj {:type :set-token-set
+                                    :set-name new-name
+                                    :token-set prev-token-set
+                                    :group? false})
+        (apply-changes-local))))
+
+(defn set-token-set
+  [changes set-name group? token-set]
+  (assert-library! changes)
+  (let [library-data   (::library-data (meta changes))
+        prev-token-set (some-> (get library-data :tokens-lib)
+                               (ctob/get-set set-name))]
+    (-> changes
+        (update :redo-changes conj {:type :set-token-set
+                                    :set-name set-name
+                                    :token-set token-set
+                                    :group? group?})
+        (update :undo-changes conj (if prev-token-set
+                                     {:type :set-token-set
+                                      :set-name (or
+                                                 ;; Undo of edit
+                                                 (:name token-set)
+                                                 ;; Undo of delete
+                                                 set-name)
+                                      :token-set prev-token-set
+                                      :group? group?}
+                                     ;; Undo of create
+                                     {:type :set-token-set
+                                      :set-name set-name
+                                      :token-set nil
+                                      :group? group?}))
         (apply-changes-local))))
 
 (defn add-component
-  ([changes id path name new-shapes updated-shapes main-instance-id main-instance-page]
-   (add-component changes id path name new-shapes updated-shapes main-instance-id main-instance-page nil))
-  ([changes id path name new-shapes updated-shapes main-instance-id main-instance-page annotation]
+  ([changes id path name updated-shapes main-instance-id main-instance-page]
+   (add-component changes id path name updated-shapes main-instance-id main-instance-page nil nil nil))
+  ([changes id path name updated-shapes main-instance-id main-instance-page annotation]
+   (add-component changes id path name updated-shapes main-instance-id main-instance-page annotation nil nil))
+  ([changes id path name updated-shapes main-instance-id main-instance-page annotation variant-id variant-properties & {:keys [apply-changes-local-library?]}]
    (assert-page-id! changes)
    (assert-objects! changes)
    (let [page-id (::page-id (meta changes))
@@ -953,8 +997,10 @@
                                       :main-instance-id main-instance-id
                                       :main-instance-page main-instance-page
                                       :annotation annotation}
-                               (some? new-shapes)  ;; this will be null in components-v2
-                               (assoc :shapes (vec new-shapes))))
+                               (some? variant-id)
+                               (assoc :variant-id variant-id)
+                               (seq variant-properties)
+                               (assoc :variant-properties variant-properties)))
                        (into (map mk-change) updated-shapes))))
          (update :undo-changes
                  (fn [undo-changes]
@@ -967,33 +1013,51 @@
                                    (map mk-change))
                              updated-shapes))))
 
-         (apply-changes-local)))))
+         (apply-changes-local {:apply-to-library? apply-changes-local-library?})))))
 
 (defn update-component
-  [changes id update-fn]
+  [changes id update-fn & {:keys [apply-changes-local-library?]}]
   (assert-library! changes)
   (let [library-data   (::library-data (meta changes))
         prev-component (get-in library-data [:components id])
         new-component  (update-fn prev-component)]
     (if prev-component
       (-> changes
-          (update :redo-changes conj {:type :mod-component
-                                      :id id
-                                      :name (:name new-component)
-                                      :path (:path new-component)
-                                      :main-instance-id (:main-instance-id new-component)
-                                      :main-instance-page (:main-instance-page new-component)
-                                      :annotation (:annotation new-component)
-                                      :objects (:objects new-component) ;; this won't exist in components-v2 (except for deleted components)
-                                      :modified-at (:modified-at new-component)})
-          (update :undo-changes conj {:type :mod-component
-                                      :id id
-                                      :name (:name prev-component)
-                                      :path (:path prev-component)
-                                      :main-instance-id (:main-instance-id prev-component)
-                                      :main-instance-page (:main-instance-page prev-component)
-                                      :annotation (:annotation prev-component)
-                                      :objects (:objects prev-component)}))
+          (update :redo-changes conj (cond-> {:type :mod-component
+                                              :id id
+                                              :name (:name new-component)
+                                              :path (:path new-component)
+                                              :main-instance-id (:main-instance-id new-component)
+                                              :main-instance-page (:main-instance-page new-component)
+                                              :annotation (:annotation new-component)
+                                              :objects (:objects new-component) ;; for deleted components
+                                              :modified-at (:modified-at new-component)}
+                                       (some? (:variant-id new-component))
+                                       (assoc :variant-id (:variant-id new-component))
+                                       (nil? (:variant-id new-component))
+                                       (dissoc :variant-id)
+                                       (seq (:variant-properties new-component))
+                                       (assoc :variant-properties (:variant-properties new-component))
+                                       (not (seq (:variant-properties new-component)))
+                                       (dissoc :variant-properties)))
+          (update :undo-changes conj (cond-> {:type :mod-component
+                                              :id id
+                                              :name (:name prev-component)
+                                              :path (:path prev-component)
+                                              :main-instance-id (:main-instance-id prev-component)
+                                              :main-instance-page (:main-instance-page prev-component)
+                                              :annotation (:annotation prev-component)
+                                              :objects (:objects prev-component)}
+                                       (some? (:variant-id prev-component))
+                                       (assoc :variant-id (:variant-id prev-component))
+                                       (nil? (:variant-id prev-component))
+                                       (dissoc :variant-id)
+                                       (seq (:variant-properties prev-component))
+                                       (assoc :variant-properties (:variant-properties prev-component))
+                                       (not (seq (:variant-properties prev-component)))
+                                       (dissoc :variant-properties)))
+          (cond-> apply-changes-local-library?
+            (apply-changes-local {:apply-to-library? true})))
       changes)))
 
 (defn delete-component
@@ -1007,7 +1071,7 @@
                                   :page-id page-id})))
 
 (defn restore-component
-  [changes id page-id main-instance]
+  [changes id page-id delta]
   (assert-library! changes)
   (-> changes
       (update :redo-changes conj {:type :restore-component
@@ -1015,7 +1079,34 @@
                                   :page-id page-id})
       (update :undo-changes conj {:type :del-component
                                   :id id
-                                  :main-instance main-instance})))
+                                  :delta delta})))
+
+(defn reorder-children
+  [changes id children]
+  (assert-page-id! changes)
+  (assert-objects! changes)
+
+  (let [page-id (::page-id (meta changes))
+        objects (lookup-objects changes)
+        shape (get objects id)
+        old-children (:shapes shape)
+
+        redo-change
+        {:type :reorder-children
+         :parent-id (:id shape)
+         :page-id page-id
+         :shapes children}
+
+        undo-change
+        {:type :reorder-children
+         :parent-id (:id shape)
+         :page-id page-id
+         :shapes old-children}]
+
+    (-> changes
+        (update :redo-changes conj redo-change)
+        (update :undo-changes conj undo-change)
+        (apply-changes-local))))
 
 (defn reorder-grid-children
   [changes ids]
@@ -1055,3 +1146,32 @@
              (reduce reorder-grid changes))]
 
     changes))
+
+(defn get-library-data
+  [changes]
+  (::library-data (meta changes)))
+
+(defn get-objects
+  [changes]
+  (dm/get-in (::file-data (meta changes)) [:pages-index uuid/zero :objects]))
+
+(defn get-page
+  [changes]
+  (::page (meta changes)))
+
+(defn get-page-id
+  [changes]
+  (::page-id (meta changes)))
+
+(defn set-base-font-size
+  [changes new-base-font-size]
+  (assert-file-data! changes)
+  (let [file-data  (::file-data (meta changes))
+        previous-font-size (ctf/get-base-font-size file-data)]
+    (-> changes
+        (update :redo-changes conj {:type :set-base-font-size
+                                    :base-font-size new-base-font-size})
+
+        (update :undo-changes conj {:type :set-base-font-size
+                                    :base-font-size previous-font-size})
+        (apply-changes-local))))

common/src/app/common/files/helpers.cljc

@@ -9,7 +9,6 @@
    [app.common.data :as d]
    [app.common.data.macros :as dm]
    [app.common.geom.shapes.common :as gco]
-   [app.common.schema :as sm]
    [app.common.uuid :as uuid]
    [clojure.set :as set]
    [clojure.walk :as walk]
@@ -201,7 +200,7 @@
         result))))
 
 (defn get-parent-seq
-  "Returns a vector of parents of the specified shape."
+  "Returns a lazy seq of parents of the specified shape."
   ([objects shape-id]
    (get-parent-seq objects (get objects shape-id) shape-id))
 
@@ -284,6 +283,22 @@
       :else
       (get-root-frame objects (:frame-id frame)))))
 
+(defn get-parent-frame
+  "Similar to `get-frame, but always return the parent frame. When root
+  frame is provided, then itself is returned."
+  [objects shape-or-id]
+  (cond
+    (map? shape-or-id)
+    (get objects (dm/get-prop shape-or-id :frame-id))
+
+    (= uuid/zero shape-or-id)
+    (get objects uuid/zero)
+
+    :else
+    (some->> shape-or-id
+             (get objects)
+             (get-frame objects))))
+
 (defn valid-frame-target?
   [objects parent-id shape-id]
   (let [shape (get objects shape-id)]
@@ -400,31 +415,57 @@
                    elements)]
     (into #{} (keep :name) elements)))
 
-(defn- extract-numeric-suffix
-  [basename]
-  (if-let [[_ p1 p2] (re-find #"(.*) ([0-9]+)$" basename)]
-    [p1 (+ 1 (d/parse-integer p2))]
-    [basename 1]))
+(defn- name-seq
+  "Creates a lazy, infinite sequence of names starting with `base-name`,
+   followed by variants with suffixes applied. The sequence follows this pattern:
+   - `base-name`
+   - `(str base-name (suffix-fn 1))`
+   - `(str base-name (suffix-fn 2))`
+   - `(str base-name (suffix-fn 3))`, etc."
+  [base-name suffix-fn]
+  (cons base-name
+        (map #(str/concat base-name (suffix-fn %))
+             (iterate inc 1))))
 
 (defn generate-unique-name
-  "A unique name generator"
-  [used basename]
+  "Generates a unique name by selecting the first available name from a generated sequence.
+   The sequence consists of `base-name` and its variants, avoiding conflicts with `existing-names`.
+
+   Parameters:
+   - `base-name` - string used as the base for name generation.
+   - `existing-names` - a collection of existing names to check for uniqueness.
+   - Options:
+     - `:suffix-fn` - a function that generates suffixes, given an integer (default: `get-suffix`).
+     - `:immediate-suffix?` - if `true`, the base name is considered taken, and suffixing starts immediately.
+
+   Returns:
+   - A unique name not present in `existing-names`."
+  [base-name existing-names & {:keys [suffix-fn immediate-suffix? suffix]}]
   (dm/assert!
    "expected a set of strings"
-   (sm/check-set-of-strings! used))
+   (coll? existing-names))
 
   (dm/assert!
    "expected a string for `basename`."
-   (string? basename))
+   (string? base-name))
+  (let [suffix-fn (if suffix-fn
+                    suffix-fn
+                    (if suffix
+                      (fn [copy-count]
+                        (str/concat "-"
+                                    suffix
+                                    (when (> copy-count 1)
+                                      (str "-" copy-count))))
+                      (fn [copy-count]
+                        (str/concat " " copy-count))))
 
-  (if-not (contains? used basename)
-    basename
-    (let [[prefix initial] (extract-numeric-suffix basename)]
-      (loop [counter initial]
-        (let [candidate (str prefix " " counter)]
-          (if (contains? used candidate)
-            (recur (inc counter))
-            candidate))))))
+        existing-name-set (cond-> (set existing-names)
+                            immediate-suffix? (conj base-name))
+        names (name-seq base-name suffix-fn)]
+
+    (->> names
+         (remove #(contains? existing-name-set %))
+         first)))
 
 (defn walk-pages
   "Go through all pages of a file and apply a function to each one"

common/src/app/common/files/migrations.cljc

@@ -16,7 +16,6 @@
    [app.common.geom.point :as gpt]
    [app.common.geom.rect :as grc]
    [app.common.geom.shapes :as gsh]
-   [app.common.geom.shapes.path :as gsp]
    [app.common.geom.shapes.text :as gsht]
    [app.common.logging :as l]
    [app.common.math :as mth]
@@ -27,7 +26,10 @@
    [app.common.types.component :as ctk]
    [app.common.types.container :as ctn]
    [app.common.types.file :as ctf]
+   [app.common.types.path :as path]
+   [app.common.types.path.segment :as path.segment]
    [app.common.types.shape :as cts]
+   [app.common.types.shape.interactions :as ctsi]
    [app.common.types.shape.shadow :as ctss]
    [app.common.uuid :as uuid]
    [clojure.set :as set]
@@ -35,9 +37,7 @@
 
 #?(:cljs (l/set-level! :info))
 
-(declare ^:private available-migrations)
-(declare ^:private migration-up-index)
-(declare ^:private migration-down-index)
+(declare available-migrations)
 
 (def version cfd/version)
 
@@ -49,7 +49,10 @@
   [file]
   (or (nil? (:version file))
       (not= cfd/version (:version file))
-      (not= available-migrations (:migrations file))))
+      (boolean
+       (->> (:migrations file #{})
+            (set/difference available-migrations)
+            (not-empty)))))
 
 (def xf:map-name
   (map :name))
@@ -96,13 +99,13 @@
                     (if (nil? migrations)
                       (generate-migrations-from-version version)
                       migrations)))
-          (migrate)
-          (update :features (fnil into #{}) (deref cfeat/*new*))
           ;; NOTE: in some future we can consider to apply
           ;; a migration to the whole database and remove
           ;; this code from this function that executes on
           ;; each file migration operation
-          (update :features cfeat/migrate-legacy-features)))))
+          (update :features cfeat/migrate-legacy-features)
+          (migrate)
+          (update :features (fnil into #{}) (deref cfeat/*new*))))))
 
 (defn migrated?
   [file]
@@ -119,16 +122,16 @@
                                (into [] shapes)
                                shapes))))
           (update-page [page]
-            (update page :objects update-vals update-object))]
+            (update page :objects d/update-vals update-object))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 (defmethod migrate-data "legacy-3"
   [data _]
   (letfn [(migrate-path [shape]
             (if-not (contains? shape :content)
-              (let [content (gsp/segments->content (:segments shape) (:close? shape))
-                    selrect (gsh/content->selrect content)
+              (let [content (path.segment/points->content (:segments shape) :close (:close? shape))
+                    selrect (path.segment/content->selrect content)
                     points  (grc/rect->points selrect)]
                 (-> shape
                     (dissoc :segments)
@@ -172,9 +175,9 @@
               (fix-empty-points)))
 
           (update-page [page]
-            (update page :objects update-vals update-object))]
+            (update page :objects d/update-vals update-object))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 ;; Put the id of the local file in :component-file in instances of
 ;; local components
@@ -187,9 +190,9 @@
               object))
 
           (update-page [page]
-            (update page :objects update-vals update-object))]
+            (update page :objects d/update-vals update-object))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 ;; Fixes issues with selrect/points for shapes with width/height =
 ;; 0 (line-like paths)
@@ -199,7 +202,7 @@
             (if (= (:type shape) :path)
               (let [{:keys [width height]} (grc/points->rect (:points shape))]
                 (if (or (mth/almost-zero? width) (mth/almost-zero? height))
-                  (let [selrect (gsh/content->selrect (:content shape))
+                  (let [selrect (path.segment/content->selrect (:content shape))
                         points (grc/rect->points selrect)
                         transform (gmt/matrix)
                         transform-inv (gmt/matrix)]
@@ -212,11 +215,11 @@
               shape))
 
           (update-container [container]
-            (update container :objects update-vals fix-line-paths))]
+            (update container :objects d/update-vals fix-line-paths))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 ;; Remove interactions pointing to deleted frames
 (defmethod migrate-data "legacy-7"
@@ -227,9 +230,9 @@
                              (filterv #(get-in page [:objects (:destination %)]) interactions))))
 
           (update-page [page]
-            (update page :objects update-vals (partial update-object page)))]
+            (update page :objects d/update-vals (partial update-object page)))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 ;; Remove groups without any shape, both in pages and components
 (defmethod migrate-data "legacy-8"
@@ -269,8 +272,8 @@
                   (assoc container :objects objects)))))]
 
     (-> data
-        (update :pages-index update-vals clean-container)
-        (update :components update-vals clean-container))))
+        (update :pages-index d/update-vals clean-container)
+        (d/update-when :components d/update-vals clean-container))))
 
 (defmethod migrate-data "legacy-9"
   [data _]
@@ -304,7 +307,7 @@
   [data _]
   (letfn [(update-page [page]
             (d/update-in-when page [:objects uuid/zero] dissoc :points :selrect))]
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 (defmethod migrate-data "legacy-11"
   [data _]
@@ -318,7 +321,7 @@
             (update page :objects (fn [objects]
                                     (update-vals objects (partial update-object objects)))))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 (defmethod migrate-data "legacy-12"
   [data _]
@@ -328,9 +331,9 @@
               (assoc :size nil)))
 
           (update-page [page]
-            (d/update-in-when page [:options :saved-grids] update-vals update-grid))]
+            (d/update-in-when page [:options :saved-grids] d/update-vals update-grid))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 ;; Add rx and ry to images
 (defmethod migrate-data "legacy-13"
@@ -348,9 +351,9 @@
               (fix-radius)))
 
           (update-page [page]
-            (update page :objects update-vals update-object))]
+            (update page :objects d/update-vals update-object))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 (defmethod migrate-data "legacy-14"
   [data _]
@@ -380,8 +383,8 @@
               container))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-16"
   [data _]
@@ -423,11 +426,11 @@
               (assign-fills)))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-17"
   [data _]
@@ -452,11 +455,11 @@
               (assoc :fills [])))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 ;; Remove position-data to solve a bug with the text positioning
 (defmethod migrate-data "legacy-18"
@@ -467,11 +470,11 @@
               (dissoc :position-data)))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-19"
   [data _]
@@ -483,11 +486,11 @@
               (dissoc :position-data)))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-25"
   [data _]
@@ -499,10 +502,10 @@
                   (update :selrect grc/make-rect)
                   (cts/create-shape))))
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-26"
   [data _]
@@ -515,11 +518,11 @@
               (assoc :transform-inverse (gmt/matrix))))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-27"
   [data _]
@@ -546,11 +549,11 @@
                   (dissoc :saved-component-root?))))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-28"
   [data _]
@@ -575,8 +578,8 @@
             (d/update-when container :objects #(update-vals % (partial update-object %))))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-29"
   [data _]
@@ -607,11 +610,11 @@
                 (update :content #(txt/transform-nodes invalid-node? fix-node %)))))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-31"
   [data _]
@@ -622,10 +625,10 @@
                   (dissoc :use-for-thumbnail?))))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-32"
   [data _]
@@ -640,11 +643,11 @@
                 object)))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-33"
   [data _]
@@ -662,9 +665,9 @@
               object))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
     (-> data
-        (update :pages-index update-vals update-container))))
+        (update :pages-index d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-34"
   [data _]
@@ -674,10 +677,10 @@
               (dissoc object :x :y :width :height)
               object))
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-36"
   [data _]
@@ -687,8 +690,8 @@
                                                   (dissoc objects nil)
                                                   objects))))]
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-37"
   [data _]
@@ -716,11 +719,11 @@
                 shape)))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-shape))]
+            (d/update-when container :objects d/update-vals update-shape))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-39"
   [data _]
@@ -738,11 +741,11 @@
               shape))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-shape))]
+            (d/update-when container :objects d/update-vals update-shape))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-40"
   [data _]
@@ -762,11 +765,11 @@
               shape))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-shape))]
+            (d/update-when container :objects d/update-vals update-shape))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-41"
   [data _]
@@ -795,11 +798,11 @@
               shape))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-shape))]
+            (d/update-when container :objects d/update-vals update-shape))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-42"
   [data _]
@@ -812,11 +815,11 @@
               object))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (def ^:private valid-fill?
   (sm/lazy-validator ::cts/fill))
@@ -841,14 +844,11 @@
               object))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
-
-(def ^:private valid-shadow?
-  (sm/lazy-validator ::ctss/shadow))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-44"
   [data _]
@@ -861,14 +861,14 @@
 
           (update-object [object]
             (let [xform (comp (map fix-shadow)
-                              (filter valid-shadow?))]
+                              (filter ctss/valid-shadow?))]
               (d/update-when object :shadow #(into [] xform %))))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-45"
   [data _]
@@ -881,9 +881,9 @@
                      :parent-id parent-id)))
 
           (update-container [container]
-            (d/update-when container :objects update-vals fix-shape))]
+            (d/update-when container :objects d/update-vals fix-shape))]
     (-> data
-        (update :pages-index update-vals update-container))))
+        (update :pages-index d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-46"
   [data _]
@@ -891,10 +891,10 @@
             (dissoc object :thumbnail))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-47"
   [data _]
@@ -915,9 +915,9 @@
                 shape)))
 
           (update-page [page]
-            (d/update-when page :objects update-vals (partial fix-shape page)))]
+            (d/update-when page :objects d/update-vals (partial fix-shape page)))]
     (-> data
-        (update :pages-index update-vals update-page))))
+        (update :pages-index d/update-vals update-page))))
 
 (defmethod migrate-data "legacy-48"
   [data _]
@@ -929,9 +929,9 @@
                 shape)))
 
           (update-page [page]
-            (d/update-when page :objects update-vals fix-shape))]
+            (d/update-when page :objects d/update-vals fix-shape))]
     (-> data
-        (update :pages-index update-vals update-page))))
+        (update :pages-index d/update-vals update-page))))
 
 ;; Remove hide-in-viewer for shapes that are origin or destination of an interaction
 (defmethod migrate-data "legacy-49"
@@ -949,9 +949,9 @@
                                     (mapcat :interactions)
                                     (map :destination)
                                     (set))]
-              (update page :objects update-vals (partial update-object destinations))))]
+              (update page :objects d/update-vals (partial update-object destinations))))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 ;; This migration mainly fixes paths with curve-to segments
 ;; without :c1x :c1y :c2x :c2y properties. Additionally, we found a
@@ -994,11 +994,11 @@
 
         update-container
         (fn [page]
-          (d/update-when page :objects update-vals update-shape))]
+          (d/update-when page :objects d/update-vals update-shape))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (def ^:private valid-color?
   (sm/lazy-validator ::ctc/color))
@@ -1018,9 +1018,9 @@
               shape))
 
           (update-page [page]
-            (d/update-when page :objects update-vals update-shape))]
+            (d/update-when page :objects d/update-vals update-shape))]
 
-    (update data :pages-index update-vals update-page)))
+    (update data :pages-index d/update-vals update-page)))
 
 
 (defmethod migrate-data "legacy-53"
@@ -1036,15 +1036,15 @@
 
           (update-shape [shape]
             (let [xform (comp (map fix-shadow)
-                              (filter valid-shadow?))]
+                              (filter ctss/valid-shadow?))]
               (d/update-when shape :shadow #(into [] xform %))))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-shape))]
+            (d/update-when container :objects d/update-vals update-shape))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 ;; This migration moves page options to the page level
 (defmethod migrate-data "legacy-55"
@@ -1096,11 +1096,11 @@
                   (update :content (partial txt/transform-nodes identity fix-fills)))))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 
 (defmethod migrate-data "legacy-57"
@@ -1127,7 +1127,7 @@
     (-> data
         (update :pages (fn [pages] (into [] (remove nil?) pages)))
         (update :pages-index dissoc nil)
-        (update :pages-index update-vals update-page))))
+        (update :pages-index d/update-vals update-page))))
 
 (defmethod migrate-data "legacy-59"
   [data _]
@@ -1138,11 +1138,11 @@
             (d/update-when shape :touched #(into #{} (map fix-touched) %)))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-shape))]
+            (d/update-when container :objects d/update-vals update-shape))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-62"
   [data _]
@@ -1175,7 +1175,7 @@
           ;; so the relevant objects are inside the component
           (d/update-when component :objects remove-cycles))]
 
-    (update data :components update-vals update-component)))
+    (d/update-when data :components d/update-vals update-component)))
 
 (defmethod migrate-data "legacy-65"
   [data _]
@@ -1186,14 +1186,14 @@
         update-page
         (fn [page]
           (-> (update-object page)
-              (update :objects update-vals update-object)))]
+              (update :objects d/update-vals update-object)))]
 
     (-> data
         (update-object)
-        (d/update-when :pages-index update-vals update-page)
-        (d/update-when :colors update-vals update-object)
-        (d/update-when :typographies update-vals update-object)
-        (d/update-when :components update-vals update-object))))
+        (update :pages-index d/update-vals update-page)
+        (d/update-when :colors d/update-vals update-object)
+        (d/update-when :typographies d/update-vals update-object)
+        (d/update-when :components d/update-vals update-object))))
 
 (defmethod migrate-data "legacy-66"
   [data _]
@@ -1207,23 +1207,122 @@
               object))
 
           (update-container [container]
-            (d/update-when container :objects update-vals update-object))]
+            (d/update-when container :objects d/update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (defmethod migrate-data "legacy-67"
   [data _]
   (letfn [(update-object [object]
             (d/update-when object :shadow #(into [] (reverse %))))
 
+          (update-container [container]
+            (d/update-when container :objects d/update-vals update-object))]
+
+    (-> data
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
+
+(defmethod migrate-data "0001-remove-tokens-from-groups"
+  [data _]
+  (letfn [(update-object [object]
+            (cond-> object
+              (and (= :group (:type object))
+                   (contains? (:applied-tokens object) :fill))
+              (assoc :fills [])
+              (and (= :group (:type object))
+                   (contains? object :applied-tokens))
+              (dissoc :applied-tokens)))
+
+          (update-page [page]
+            (d/update-when page :objects d/update-vals update-object))]
+
+    (update data :pages-index d/update-vals update-page)))
+
+(defmethod migrate-data "0002-clean-shape-interactions"
+  [data _]
+  (let [decode-fn   (sm/decoder ctsi/schema:interaction sm/json-transformer)
+        validate-fn (sm/validator ctsi/schema:interaction)
+
+        xform
+        (comp
+         (map decode-fn)
+         (filter validate-fn))
+
+        update-object
+        (fn [object]
+          (d/update-when object :interactions
+                         (fn [interactions]
+                           (into [] xform interactions))))
+
+        update-container
+        (fn [container]
+          (d/update-when container :objects d/update-vals update-object))]
+
+    (-> data
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
+
+(defmethod migrate-data "0002-normalize-bool-content"
+  [data _]
+  (letfn [(update-object [object]
+            ;; NOTE: we still preserve the previous value for possible
+            ;; rollback, we still need to perform an other migration
+            ;; for properly delete the bool-content prop from shapes
+            ;; once the know the migration was OK
+            (if (cfh/bool-shape? object)
+              (if-let [content (:bool-content object)]
+                (assoc object :content content)
+                object)
+              (dissoc object :bool-content :bool-type)))
+
           (update-container [container]
             (d/update-when container :objects update-vals update-object))]
 
     (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
+
+(defmethod migrate-data "0003-fix-root-shape"
+  [data _]
+  (letfn [(update-object [shape]
+            (if (= (:id shape) uuid/zero)
+              (-> shape
+                  (assoc :parent-id uuid/zero)
+                  (assoc :frame-id uuid/zero)
+                  ;; We explicitly dissoc them and let the shape-setup
+                  ;; to regenerate it with valid values.
+                  (dissoc :selrect)
+                  (dissoc :points)
+                  (cts/setup-shape))
+              shape))
+
+          (update-container [container]
+            (d/update-when container :objects d/update-vals update-object))]
+
+    (-> data
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container)
+        (d/without-nils))))
+
+(defmethod migrate-data "0003-convert-path-content"
+  [data _]
+  (some-> cfeat/*new* (swap! conj "fdata/path-data"))
+
+  (letfn [(update-object [object]
+            (if (or (cfh/bool-shape? object)
+                    (cfh/path-shape? object))
+              (update object :content path/content)
+              object))
+
+          (update-container [container]
+            (d/update-when container :objects update-vals update-object))]
+
+    (-> data
+        (update :pages-index d/update-vals update-container)
+        (d/update-when :components d/update-vals update-container))))
 
 (def available-migrations
   (into (d/ordered-set)
@@ -1278,4 +1377,9 @@
          "legacy-62"
          "legacy-65"
          "legacy-66"
-         "legacy-67"]))
+         "legacy-67"
+         "0001-remove-tokens-from-groups"
+         "0002-normalize-bool-content"
+         "0002-clean-shape-interactions"
+         "0003-fix-root-shape"
+         "0003-convert-path-content"]))

common/src/app/common/files/repair.cljc

@@ -572,6 +572,51 @@
         (pcb/with-file-data file-data)
         (pcb/update-shapes [(:id shape)] repair-shape))))
 
+(defmethod repair-error :not-a-variant
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :invalid-variant-id
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :invalid-variant-properties
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :variant-not-main
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :parent-not-variant
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :variant-bad-name
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :variant-no-properties
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :variant-bad-variant-name
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
+(defmethod repair-error :variant-component-bad-name
+  [_ error file _]
+  (log/error :hint "Variant error code, we don't want to auto repair it for now" :code (:code error))
+  file)
+
 (defmethod repair-error :default
   [_ error file _]
   (log/error :hint "Unknown error code, don't know how to repair" :code (:code error))

common/src/app/common/files/shapes_builder.cljc

@@ -4,7 +4,7 @@
 ;;
 ;; Copyright (c) KALEIDOS INC
 
-(ns app.common.svg.shapes-builder
+(ns app.common.files.shapes-builder
   "A SVG to Shapes builder."
   (:require
    [app.common.colors :as clr]
@@ -21,7 +21,8 @@
    [app.common.math :as mth]
    [app.common.schema :as sm :refer [max-safe-int min-safe-int]]
    [app.common.svg :as csvg]
-   [app.common.svg.path :as path]
+   [app.common.types.path :as path]
+   [app.common.types.path.segment :as path.segm]
    [app.common.types.shape :as cts]
    [app.common.uuid :as uuid]
    [cuerdas.core :as str]))
@@ -218,11 +219,11 @@
 (defn create-path-shape [name frame-id svg-data {:keys [attrs] :as data}]
   (when (and (contains? attrs :d) (seq (:d attrs)))
     (let [transform (csvg/parse-transform (:transform attrs))
-          content   (cond-> (path/parse (:d attrs))
+          content   (cond-> (path/from-string (:d attrs))
                       (some? transform)
-                      (gsh/transform-content transform))
+                      (path.segm/transform-content transform))
 
-          selrect    (gsh/content->selrect content)
+          selrect    (path.segm/content->selrect content)
           points     (grc/rect->points selrect)
           origin     (gpt/negate (gpt/point svg-data))
           attrs      (-> (dissoc attrs :d :transform)
@@ -435,16 +436,12 @@
 
         attrs
         (-> attrs
-            (cond-> linecap
-              (dissoc :strokeLinecap))
             (cond-> (some? color)
               (dissoc :stroke :strokeWidth :strokeOpacity))
             (update
              :style
              (fn [style]
                (-> style
-                   (cond-> linecap
-                     (dissoc :strokeLinecap))
                    (cond-> (some? color)
                      (dissoc :stroke :strokeWidth :strokeOpacity)))))
             (d/without-nils))]
@@ -461,12 +458,14 @@
 
       (and (some? linecap) (cfh/path-shape? shape)
            (or (= linecap :round) (= linecap :square)))
+
       (assoc :stroke-cap-start linecap
-             :stroke-cap-end linecap)
+             :stroke-cap-end linecap
+             :stroke-linecap linecap)
 
       (d/any-key? (dm/get-in shape [:strokes 0])
                   :strokeColor :strokeOpacity :strokeWidth
-                  :strokeCapStart :strokeCapEnd)
+                  :strokeLinecap :strokeCapStart :strokeCapEnd)
       (assoc-in [:strokes 0 :stroke-style] :svg))))
 
 (defn setup-opacity [shape]

common/src/app/common/files/shapes_helpers.cljc

@@ -15,6 +15,8 @@
    [app.common.types.shape.layout :as ctl]
    [app.common.uuid :as uuid]))
 
+;; FIXME: move to logic?
+
 (defn prepare-add-shape
   [changes shape objects]
   (let [index   (:index (meta shape))
@@ -35,18 +37,21 @@
                       (pcb/update-shapes [(:parent-id shape)] #(ctl/push-into-cell % [id] row column)))
                     (cond-> (ctl/grid-layout? objects (:parent-id shape))
                       (pcb/update-shapes [(:parent-id shape)] ctl/assign-cells {:with-objects? true})))]
+
     [shape changes]))
 
 (defn prepare-move-shapes-into-frame
-  [changes frame-id shapes objects]
+  [changes frame-id shapes objects remove-layout-data?]
   (let [parent-id  (dm/get-in objects [frame-id :parent-id])
         shapes     (remove #(= % parent-id) shapes)
         to-move    (->> shapes
                         (map (d/getf objects))
                         (not-empty))]
+
     (if to-move
       (-> changes
-          (cond-> (not (ctl/any-layout? objects frame-id))
+          (cond-> (and remove-layout-data?
+                       (not (ctl/any-layout? objects frame-id)))
             (pcb/update-shapes shapes ctl/remove-layout-item-data))
           (pcb/update-shapes shapes #(cond-> % (cfh/frame-shape? %) (assoc :hide-in-viewer true)))
           (pcb/change-parent frame-id to-move 0)
@@ -61,6 +66,10 @@
     changes id parent-id objects selected index frame-name without-fill? nil))
 
   ([changes id parent-id objects selected index frame-name without-fill? target-cell-id]
+   (prepare-create-artboard-from-selection
+    changes id parent-id objects selected index frame-name without-fill? target-cell-id nil))
+
+  ([changes id parent-id objects selected index frame-name without-fill? target-cell-id delta]
    (when-let [selected-objs (->> selected
                                  (map (d/getf objects))
                                  (not-empty))]
@@ -98,10 +107,11 @@
                     :id))
              target-cell-id)
 
+
            attrs
            {:type :frame
-            :x (:x srect)
-            :y (:y srect)
+            :x (cond-> (:x srect) delta (+ (:x delta)))
+            :y (cond-> (:y srect) delta (+ (:y delta)))
             :width (:width srect)
             :height (:height srect)}
 
@@ -133,7 +143,7 @@
            (prepare-add-shape changes shape objects)
 
            changes
-           (prepare-move-shapes-into-frame changes (:id shape) selected' objects)
+           (prepare-move-shapes-into-frame changes (:id shape) selected' objects false)
 
            changes
            (cond-> changes

common/src/app/common/files/tokens.cljc

@@ -1,7 +1,13 @@
-(ns app.main.ui.workspace.tokens.token
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.common.files.tokens
   (:require
    [app.common.data :as d]
-   [app.main.ui.workspace.tokens.tinycolor :as tinycolor]
+   [app.common.data.macros :as dm]
    [clojure.set :as set]
    [cuerdas.core :as str]))
 
@@ -21,7 +27,9 @@
                         {:value parsed-value
                          :unit unit}))))
 
-(defn token-identifier [{:keys [name] :as _token}]
+;; FIXME: looks very redundant function
+(defn token-identifier
+  [{:keys [name] :as _token}]
   name)
 
 (defn attributes-map
@@ -43,7 +51,7 @@
 (defn token-attribute-applied?
   "Test if `token` is applied to a `shape` on single `token-attribute`."
   [token shape token-attribute]
-  (when-let [id (get-in shape [:applied-tokens token-attribute])]
+  (when-let [id (dm/get-in shape [:applied-tokens token-attribute])]
     (= (token-identifier token) id)))
 
 (defn token-applied?
@@ -56,15 +64,18 @@
   [token shapes token-attributes]
   (some #(token-applied? token % token-attributes) shapes))
 
-(defn shapes-ids-by-applied-attributes [token shapes token-attributes]
-  (reduce (fn [acc shape]
-            (let [applied-ids-by-attribute (->> (map #(when (token-attribute-applied? token shape %)
-                                                        [% #{(:id shape)}])
-                                                     token-attributes)
-                                                (filter some?)
-                                                (into {}))]
-              (merge-with into acc applied-ids-by-attribute)))
-          {} shapes))
+(defn shapes-ids-by-applied-attributes
+  [token shapes token-attributes]
+  (let [conj* (fnil conj #{})]
+    (reduce (fn [result shape]
+              (let [shape-id (dm/get-prop shape :id)]
+                (->> token-attributes
+                     (filter #(token-attribute-applied? token shape %))
+                     (reduce (fn [result attr]
+                               (update result attr conj* shape-id))
+                             result))))
+            {}
+            shapes)))
 
 (defn shapes-applied-all? [ids-by-attributes shape-ids attributes]
   (every? #(set/superset? (get ids-by-attributes %) shape-ids) attributes))
@@ -122,13 +133,6 @@
 (defn color-token? [token]
   (= (:type token) :color))
 
-(defn color-bullet-color [token-color-value]
-  (when-let [tc (tinycolor/valid-color token-color-value)]
-    (if (tinycolor/alpha tc)
-      {:color (tinycolor/->hex-string tc)
-       :opacity (tinycolor/alpha tc)}
-      (tinycolor/->hex-string tc))))
-
-(defn resolved-token-bullet-color [{:keys [resolved-value] :as token}]
-  (when (and resolved-value (color-token? token))
-    (color-bullet-color resolved-value)))
+;; FIXME: this should be precalculated ?
+(defn is-reference? [token]
+  (str/includes? (:value token) "{"))

common/src/app/common/files/validate.cljc

@@ -10,12 +10,15 @@
    [app.common.data.macros :as dm]
    [app.common.exceptions :as ex]
    [app.common.files.helpers :as cfh]
+   [app.common.files.variant :as cfv]
    [app.common.schema :as sm]
    [app.common.types.component :as ctk]
+   [app.common.types.components-list :as ctkl]
    [app.common.types.container :as ctn]
    [app.common.types.file :as ctf]
    [app.common.types.pages-list :as ctpl]
    [app.common.types.shape-tree :as ctst]
+   [app.common.types.variant :as ctv]
    [app.common.uuid :as uuid]
    [cuerdas.core :as str]))
 
@@ -56,7 +59,17 @@
     :instance-head-not-frame
     :misplaced-slot
     :missing-slot
-    :shape-ref-cycle})
+    :shape-ref-cycle
+    :not-a-variant
+    :invalid-variant-id
+    :invalid-variant-properties
+    :variant-not-main
+    :parent-not-variant
+    :variant-bad-name
+    :variant-bad-variant-name
+    :variant-component-bad-name
+    :variant-no-properties
+    :variant-component-bad-id})
 
 (def ^:private schema:error
   [:map {:title "ValidationError"}
@@ -401,6 +414,68 @@
   (check-empty-swap-slot shape file page)
   (run! #(check-shape % file page libraries :context :not-component) (:shapes shape)))
 
+(defn- check-variant-container
+  "Shape is a variant container, so:
+     -all its children should be variants with variant-id equals to the shape-id
+     -all the components should have the same properties
+   "
+  [shape file page]
+  (let [shape-id   (:id shape)
+        shapes     (:shapes shape)
+        children   (map #(ctst/get-shape page %) shapes)
+        prop-names (cfv/extract-properties-names (first children) (:data file))]
+    (doseq [child children]
+      (if (not (ctk/is-variant? child))
+        (report-error :not-a-variant
+                      (str/ffmt "Shape % should be a variant" (:id child))
+                      child file page)
+        (do
+          (when (not= (:variant-id child) shape-id)
+            (report-error :invalid-variant-id
+                          (str/ffmt "Variant % has invalid variant-id %" (:id child) (:variant-id child))
+                          child file page))
+          (when (not= prop-names (cfv/extract-properties-names child (:data file)))
+            (report-error :invalid-variant-properties
+                          (str/ffmt "Variant % has invalid properties %" (:id child) (vec prop-names))
+                          child file page)))))))
+
+(defn- check-variant
+  "Shape is a variant, so
+     -it should be a main component
+     -its parent should be a variant-container
+     -its variant-name is derived from the properties
+     -its name should be tha same as its parent's
+   "
+  [shape file page]
+  (let [parent    (ctst/get-shape page (:parent-id shape))
+        component (ctkl/get-component (:data file) (:component-id shape) true)
+        name      (ctv/properties-to-name (:variant-properties component))]
+    (when-not (ctk/main-instance? shape)
+      (report-error :variant-not-main
+                    (str/ffmt "Variant % is not a main instance" (:id shape))
+                    shape file page))
+    (when-not (ctk/is-variant-container? parent)
+      (report-error :parent-not-variant
+                    (str/ffmt "Variant % has an invalid parent" (:id shape))
+                    shape file page))
+
+    (when-not (= name (:variant-name shape))
+      (report-error :variant-bad-variant-name
+                    (str/ffmt "Variant % has an invalid variant-name" (:id shape))
+                    shape file page))
+    (when-not (= (:name parent) (:name shape))
+      (report-error :variant-bad-name
+                    (str/ffmt "Variant % has an invalid name" (:id shape))
+                    shape file page))
+    (when-not (= (:name parent) (cfh/merge-path-item (:path component) (:name component)))
+      (report-error :variant-component-bad-name
+                    (str/ffmt "Component % has an invalid name" (:id shape))
+                    shape file page))
+    (when-not (= (:variant-id component) (:variant-id shape))
+      (report-error :variant-component-bad-id
+                    (str/ffmt "Variant % has adifferent variant-id than its component" (:id shape))
+                    shape file page))))
+
 (defn- check-shape
   "Validate referential integrity and semantic coherence of
   a shape and all its children. Report all errors found.
@@ -421,6 +496,12 @@
       (check-parent-children shape file page)
       (check-frame shape file page)
 
+      (when (ctk/is-variant-container? shape)
+        (check-variant-container shape file page))
+
+      (when (ctk/is-variant? shape)
+        (check-variant shape file page))
+
       (if (ctk/instance-head? shape)
         (if (not= :frame (:type shape))
           (report-error :instance-head-not-frame
@@ -496,6 +577,24 @@
                     "This deleted component has shapes with shape-ref pointing to self"
                     component file nil :cycles-ids cycles-ids))))
 
+(defn- check-variant-component
+  "Component is a variant, so:
+     -Its main should be a variant
+     -It should have at least one variant property"
+  [component file]
+  (let [component-page (ctf/get-component-page (:data file) component)
+        main-component (if (:deleted component)
+                         (dm/get-in component [:objects (:main-instance-id component)])
+                         (ctst/get-shape component-page (:main-instance-id component)))]
+    (when-not (ctk/is-variant? main-component)
+      (report-error :not-a-variant
+                    (str/ffmt "Shape % should be a variant" (:id main-component))
+                    main-component file component-page))
+    (when (< (count (:variant-properties component)) 1)
+      (report-error :variant-no-properties
+                    (str/ffmt "Component variant % should have properties" (:id main-component))
+                    main-component file nil))))
+
 (defn- check-component
   "Validate semantic coherence of a component. Report all errors found."
   [component file]
@@ -505,7 +604,10 @@
                   component file nil))
   (when (:deleted component)
     (check-component-duplicate-swap-slot component file)
-    (check-ref-cycles component file)))
+    (check-ref-cycles component file))
+
+  (when (ctk/is-variant? component)
+    (check-variant-component component file)))
 
 (defn- get-orphan-shapes
   [{:keys [objects] :as page}]

common/src/app/common/files/variant.cljc

@@ -0,0 +1,84 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+(ns app.common.files.variant
+  (:require
+   [app.common.data.macros :as dm]
+   [app.common.types.component :as ctc]
+   [app.common.types.components-list :as ctcl]
+   [app.common.types.variant :as ctv]
+   [cuerdas.core :as str]))
+
+
+(defn find-variant-components
+  "Find a list of the components thet belongs to this variant-id"
+  [data objects variant-id]
+  ;; We can't simply filter components, because we need to maintain the order
+  (->> (dm/get-in objects [variant-id :shapes])
+       (map #(dm/get-in objects [% :component-id]))
+       (map #(ctcl/get-component data % true))
+       reverse))
+
+(defn- dashes-to-end
+  [property-values]
+  (let [dashes (if (some #(= % "--") property-values) ["--"] [])]
+    (concat (remove #(= % "--") property-values) dashes)))
+
+
+(defn extract-properties-names
+  [shape data]
+  (->> shape
+       (#(ctcl/get-component data (:component-id %) true))
+       :variant-properties
+       (map :name)))
+
+
+(defn extract-properties-values
+  [data objects variant-id]
+  (->> (find-variant-components data objects variant-id)
+       (mapcat :variant-properties)
+       (group-by :name)
+       (map (fn [[k v]]
+              {:name k
+               :value (->> v
+                           (map #(if (str/empty? (:value %)) "--" (:value %)))
+                           distinct
+                           dashes-to-end)}))))
+
+(defn get-variant-mains
+  [component data]
+  (assert (ctv/valid-variant-component? component) "expected valid component variant")
+  (when-let [variant-id (:variant-id component)]
+    (let [page-id (:main-instance-page component)
+          objects (-> (dm/get-in data [:pages-index page-id])
+                      (get :objects))]
+      (dm/get-in objects [variant-id :shapes]))))
+
+
+(defn is-secondary-variant?
+  [component data]
+  (let [shapes  (get-variant-mains component data)]
+    (and (seq shapes)
+         (not= (:main-instance-id component) (last shapes)))))
+
+(defn get-primary-variant
+  [data component]
+  (let [page-id    (:main-instance-page component)
+        objects    (-> (dm/get-in data [:pages-index page-id])
+                       (get :objects))
+        variant-id (:variant-id component)]
+    (->> (dm/get-in objects [variant-id :shapes])
+         peek
+         (get objects))))
+
+(defn get-primary-component
+  [data component-id]
+  (when-let [component (ctcl/get-component data component-id)]
+    (if (ctc/is-variant? component)
+      (->> component
+           (get-primary-variant data)
+           :component-id
+           (ctcl/get-component data))
+      component)))

common/src/app/common/flags.cljc

@@ -79,7 +79,7 @@
     :file-schema-validation
     ;; Reports the schema validation errors internally.
     :soft-file-schema-validation
-    ;; Activates the referential integrity validation during update file; related to components-v2.
+    ;; Activates the referential integrity validation during update file.
     :file-validation
     ;; Reports the referential integrity validation errors internally.
     :soft-file-validation
@@ -124,7 +124,9 @@
     ;; TODO: deprecate this flag and consolidate the code
     :export-file-v3
     :render-wasm-dpr
-    :hide-release-modal})
+    :hide-release-modal
+    :subscriptions
+    :subscriptions-old})
 
 (def all-flags
   (set/union email login varia))

common/src/app/common/geom/matrix.cljc

@@ -126,21 +126,20 @@
       o)))
 
 (def schema:matrix
-  {:type :map
-   :pred valid-matrix?
-   :type-properties
-   {:title "matrix"
-    :description "Matrix instance"
-    :error/message "expected a valid matrix instance"
-    :gen/gen (matrix-generator)
-    :decode/json decode-matrix
-    :decode/string decode-matrix
-    :encode/json matrix->json
-    :encode/string matrix->str
-    ::oapi/type "string"
-    ::oapi/format "matrix"}})
-
-(sm/register! ::matrix schema:matrix)
+  (sm/register!
+   {:type ::matrix
+    :pred valid-matrix?
+    :type-properties
+    {:title "matrix"
+     :description "Matrix instance"
+     :error/message "expected a valid matrix instance"
+     :gen/gen (matrix-generator)
+     :decode/json decode-matrix
+     :decode/string decode-matrix
+     :encode/json matrix->json
+     :encode/string matrix->str
+     ::oapi/type "string"
+     ::oapi/format "matrix"}}))
 
 ;; FIXME: deprecated
 (s/def ::a ::us/safe-float)

common/src/app/common/geom/point.cljc

@@ -5,7 +5,7 @@
 ;; Copyright (c) KALEIDOS INC
 
 (ns app.common.geom.point
-  (:refer-clojure :exclude [divide min max abs])
+  (:refer-clojure :exclude [divide min max abs zero?])
   (:require
    #?(:clj [app.common.fressian :as fres])
    #?(:cljs [cljs.core :as c]
@@ -85,24 +85,22 @@
     (into {} p)
     p))
 
-;; FIXME: make like matrix
 (def schema:point
-  {:type ::point
-   :pred valid-point?
-   :type-properties
-   {:title "point"
-    :description "Point"
-    :error/message "expected a valid point"
-    :gen/gen (->> (sg/tuple (sg/small-int) (sg/small-int))
-                  (sg/fmap #(apply pos->Point %)))
-    ::oapi/type "string"
-    ::oapi/format "point"
-    :decode/json decode-point
-    :decode/string decode-point
-    :encode/json point->json
-    :encode/string point->str}})
-
-(sm/register! schema:point)
+  (sm/register!
+   {:type ::point
+    :pred valid-point?
+    :type-properties
+    {:title "point"
+     :description "Point"
+     :error/message "expected a valid point"
+     :gen/gen (->> (sg/tuple (sg/small-int) (sg/small-int))
+                   (sg/fmap #(apply pos->Point %)))
+     ::oapi/type "string"
+     ::oapi/format "point"
+     :decode/json decode-point
+     :decode/string decode-point
+     :encode/json point->json
+     :encode/string point->str}}))
 
 (defn point-like?
   [{:keys [x y] :as v}]
@@ -470,6 +468,13 @@
   (and ^boolean (mth/almost-zero? (dm/get-prop p :x))
        ^boolean (mth/almost-zero? (dm/get-prop p :y))))
 
+(defn zero?
+  [p]
+  (let [x (dm/get-prop p :x)
+        y (dm/get-prop p :y)]
+    (and ^boolean (== 0 x)
+         ^boolean (== 0 y))))
+
 (defn lerp
   "Calculates a linear interpolation between two points given a tvalue"
   [p1 p2 t]

common/src/app/common/geom/shapes.cljc

@@ -10,13 +10,11 @@
    [app.common.data.macros :as dm]
    [app.common.geom.point :as gpt]
    [app.common.geom.rect :as grc]
-   [app.common.geom.shapes.bool :as gsb]
    [app.common.geom.shapes.common :as gco]
    [app.common.geom.shapes.constraints :as gct]
    [app.common.geom.shapes.corners :as gsc]
    [app.common.geom.shapes.fit-frame :as gsff]
    [app.common.geom.shapes.intersect :as gsi]
-   [app.common.geom.shapes.path :as gsp]
    [app.common.geom.shapes.transforms :as gtr]
    [app.common.math :as mth]))
 
@@ -166,7 +164,7 @@
 (dm/export gtr/calculate-geometry)
 (dm/export gtr/update-group-selrect)
 (dm/export gtr/update-mask-selrect)
-(dm/export gtr/update-bool-selrect)
+(dm/export gtr/update-bool)
 (dm/export gtr/apply-transform)
 (dm/export gtr/transform-shape)
 (dm/export gtr/transform-selrect)
@@ -180,12 +178,6 @@
 ;; Constratins
 (dm/export gct/calc-child-modifiers)
 
-;; PATHS
-;; FIXME: rename
-(dm/export gsp/content->selrect)
-(dm/export gsp/transform-content)
-(dm/export gsp/open-path?)
-
 ;; Intersection
 (dm/export gsi/overlaps?)
 (dm/export gsi/overlaps-path?)
@@ -193,9 +185,6 @@
 (dm/export gsi/has-point-rect?)
 (dm/export gsi/rect-contains-shape?)
 
-;; Bool
-(dm/export gsb/calc-bool-content)
-
 ;; Constraints
 (dm/export gct/default-constraints-h)
 (dm/export gct/default-constraints-v)
@@ -206,6 +195,7 @@
 
 ;; Rect
 (dm/export grc/rect->points)
+(dm/export grc/center->rect)
 
 ;;
 (dm/export gsff/fit-frame-modifiers)

common/src/app/common/geom/shapes/bool.cljc

@@ -1,29 +0,0 @@
-;; This Source Code Form is subject to the terms of the Mozilla Public
-;; License, v. 2.0. If a copy of the MPL was not distributed with this
-;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
-;;
-;; Copyright (c) KALEIDOS INC
-
-(ns app.common.geom.shapes.bool
-  (:require
-   [app.common.data :as d]
-   [app.common.files.helpers :as cpf]
-   [app.common.svg.path.bool :as pb]
-   [app.common.svg.path.shapes-to-path :as stp]))
-
-(defn calc-bool-content
-  [shape objects]
-
-  (let [extract-content-xf
-        (comp (map (d/getf objects))
-              (filter (comp not :hidden))
-              (remove cpf/svg-raw-shape?)
-              (map #(stp/convert-to-path % objects))
-              (map :content))
-
-        shapes-content
-        (into [] extract-content-xf (:shapes shape))]
-    (pb/content-bool (:bool-type shape) shapes-content)))
-
-
-

common/src/app/common/geom/shapes/bounds.cljc

@@ -10,8 +10,8 @@
    [app.common.data.macros :as dm]
    [app.common.files.helpers :as cfh]
    [app.common.geom.rect :as grc]
-   [app.common.geom.shapes.path :as gsp]
-   [app.common.math :as mth]))
+   [app.common.math :as mth]
+   [app.common.types.path :as path]))
 
 (defn shape-stroke-margin
   [shape stroke-width]
@@ -104,7 +104,7 @@
    (let [strokes (:strokes shape)
 
          open-path?    (and ^boolean (cfh/path-shape? shape)
-                            ^boolean (gsp/open-path? shape))
+                            ^boolean (path/shape-with-open-path? shape))
 
          stroke-width
          (->> strokes

common/src/app/common/geom/shapes/grid_layout/layout_data.cljc

@@ -39,7 +39,7 @@
 ;;
 ;;   5. If any track still has an infinite growth limit set its growth limit to its base size.
 
-;;   - Distribute extra space accross spaned tracks
+;; - Distribute extra space accross spaned tracks
 ;; - Maximize tracks
 ;;
 ;; - Expand flexible tracks
@@ -198,7 +198,7 @@
 
     track-list))
 
-(defn add-auto-size
+(defn stretch-tracks
   [track-list add-size]
   (->> track-list
        (mapv (fn [{:keys [type size max-size] :as track}]
@@ -357,7 +357,8 @@
                       to-idx (+ (dec (get cell prop)) (get cell prop-span))
                       indexed-tracks (subvec (d/enumerate track-list) from-idx to-idx)
 
-                      to-allocate (size-to-allocate type parent (get children-map shape-id) cell bounds objects)
+                      to-allocate
+                      (size-to-allocate type parent (get children-map shape-id) cell bounds objects)
 
                       ;; Remove the size and the tracks that are not allocated
                       [to-allocate total-frs indexed-tracks]
@@ -493,11 +494,11 @@
 
          column-tracks (cond-> column-tracks
                          (= :stretch (:layout-justify-content parent))
-                         (add-auto-size column-add-auto))
+                         (stretch-tracks column-add-auto))
 
          row-tracks    (cond-> row-tracks
                          (= :stretch (:layout-align-content parent))
-                         (add-auto-size row-add-auto))
+                         (stretch-tracks row-add-auto))
 
          column-total-size (tracks-total-size column-tracks)
          row-total-size    (tracks-total-size row-tracks)

common/src/app/common/geom/shapes/intersect.cljc

@@ -13,9 +13,9 @@
    [app.common.geom.point :as gpt]
    [app.common.geom.rect :as grc]
    [app.common.geom.shapes.common :as gco]
-   [app.common.geom.shapes.path :as gpp]
    [app.common.geom.shapes.text :as gte]
-   [app.common.math :as mth]))
+   [app.common.math :as mth]
+   [app.common.types.path.segment :as path.segm]))
 
 (defn orientation
   "Given three ordered points gives the orientation
@@ -186,7 +186,7 @@
           rect-lines   (points->lines rect-points)
           path-lines   (if simple?
                          (points->lines (:points shape))
-                         (gpp/path->lines shape))
+                         (path.segm/path->lines shape))
           start-point (-> shape :content (first) :params (gpt/point))]
 
       (or (intersects-lines? rect-lines path-lines)

common/src/app/common/geom/shapes/transforms.cljc

@@ -12,11 +12,10 @@
    [app.common.geom.matrix :as gmt]
    [app.common.geom.point :as gpt]
    [app.common.geom.rect :as grc]
-   [app.common.geom.shapes.bool :as gshb]
    [app.common.geom.shapes.common :as gco]
-   [app.common.geom.shapes.path :as gpa]
    [app.common.math :as mth]
-   [app.common.types.modifiers :as ctm]))
+   [app.common.types.modifiers :as ctm]
+   [app.common.types.path :as path]))
 
 #?(:clj (set! *warn-on-reflection* true))
 
@@ -77,7 +76,11 @@
               position-data)
         position-data))))
 
-;; FIXME: revist usage of mutability
+;; FIXME: review performance of this; this function is executing too
+;; many times, including when the point vector is 0,0. This function
+;; can be implemented in function of transform which is already mor
+;; performant
+
 (defn move
   "Move the shape relatively to its current
   position applying the provided delta."
@@ -95,8 +98,8 @@
         (d/update-when :x d/safe+ dx)
         (d/update-when :y d/safe+ dy)
         (d/update-when :position-data move-position-data mvec)
-        (cond-> (= :bool type) (update :bool-content gpa/move-content mvec))
-        (cond-> (= :path type) (update :content gpa/move-content mvec)))))
+        (cond-> (or (= :bool type) (= :path type))
+          (update :content path/move-content mvec)))))
 
 ;; --- Absolute Movement
 
@@ -317,14 +320,11 @@
         points  (gco/transform-points  (dm/get-prop shape :points) transform-mtx)
         selrect (gco/transform-selrect (dm/get-prop shape :selrect) transform-mtx)
 
-        shape   (if (= type :bool)
-                  (update shape :bool-content gpa/transform-content transform-mtx)
-                  shape)
         shape   (if (= type :text)
                   (update shape :position-data transform-position-data transform-mtx)
                   shape)
-        shape   (if (= type :path)
-                  (update shape :content gpa/transform-content transform-mtx)
+        shape   (if (or (= type :path) (= type :bool))
+                  (update shape :content path/transform-content transform-mtx)
                   (assoc shape
                          :x (dm/get-prop selrect :x)
                          :y (dm/get-prop selrect :y)
@@ -355,12 +355,9 @@
             rotation (mod (+ (d/nilv (:rotation shape) 0)
                              (d/nilv (dm/get-in shape [:modifiers :rotation]) 0))
                           360)
-            shape    (if (= type :bool)
-                       (update shape :bool-content gpa/transform-content transform-mtx)
-                       shape)
 
-            shape    (if (= type :path)
-                       (update shape :content gpa/transform-content transform-mtx)
+            shape    (if (or (= type :path) (= type :bool))
+                       (update shape :content path/transform-content transform-mtx)
                        (assoc shape
                               :x (dm/get-prop selrect :x)
                               :y (dm/get-prop selrect :y)
@@ -377,8 +374,14 @@
   "Given a new set of points transformed, set up the rectangle so it keeps
   its properties. We adjust de x,y,width,height and create a custom transform"
   [shape transform-mtx]
-  (if ^boolean (gmt/move? transform-mtx)
+  (cond
+    (nil? transform-mtx)
+    shape
+
+    ^boolean (gmt/move? transform-mtx)
     (apply-transform-move shape transform-mtx)
+
+    :else
     (apply-transform-generic shape transform-mtx)))
 
 (defn- update-group-viewbox
@@ -450,20 +453,14 @@
         (assoc :flip-x  (-> mask :flip-x))
         (assoc :flip-y  (-> mask :flip-y)))))
 
-(defn update-bool-selrect
+(defn update-bool
   "Calculates the selrect+points for the boolean shape"
-  [shape children objects]
-
-  (let [bool-content     (gshb/calc-bool-content shape objects)
-        shape            (assoc shape :bool-content bool-content)
-        [points selrect] (gpa/content->points+selrect shape bool-content)]
-
-    (if (and (some? selrect) (d/not-empty? points))
-      (-> shape
-          (assoc :selrect selrect)
-          (assoc :points points))
-      (update-group-selrect shape children))))
+  [shape objects]
+  (let [content (path/calc-bool-content shape objects)
+        shape   (assoc shape :content content)]
+    (path/update-geometry shape)))
 
+;; FIXME: revisit
 (defn update-shapes-geometry
   [objects ids]
   (->> ids
@@ -477,7 +474,7 @@
                   (update-mask-selrect shape children)
 
                   (cfh/bool-shape? shape)
-                  (update-bool-selrect shape children objects)
+                  (update-bool shape objects)
 
                   (cfh/group-shape? shape)
                   (update-group-selrect shape children)

common/src/app/common/logic/shapes.cljc

@@ -10,8 +10,10 @@
    [app.common.files.changes-builder :as pcb]
    [app.common.files.helpers :as cfh]
    [app.common.geom.shapes :as gsh]
+   [app.common.logic.variant-properties :as clvp]
    [app.common.types.component :as ctk]
    [app.common.types.container :as ctn]
+   [app.common.types.pages-list :as ctpl]
    [app.common.types.shape.interactions :as ctsi]
    [app.common.types.shape.layout :as ctl]
    [app.common.types.token :as cto]
@@ -79,181 +81,189 @@
         (pcb/update-shapes ids update-fn {:attrs #{:blocked :hidden}}))))
 
 (defn generate-delete-shapes
-  [changes file page objects ids {:keys [components-v2 ignore-touched component-swap]}]
-  (let [ids           (cfh/clean-loops objects ids)
+  ([changes file page objects ids options]
+   (generate-delete-shapes (-> changes
+                               (pcb/with-page page)
+                               (pcb/with-objects objects)
+                               (pcb/with-library-data file))
+                           ids
+                           options))
+  ([changes ids {:keys [ignore-touched component-swap]}]
+   (let [objects (pcb/get-objects changes)
+         data    (pcb/get-library-data changes)
+         page-id (pcb/get-page-id changes)
+         page    (or (pcb/get-page changes)
+                     (ctpl/get-page data page-id))
 
-        in-component-copy?
-        (fn [shape-id]
+         ids     (cfh/clean-loops objects ids)
+         in-component-copy?
+         (fn [shape-id]
           ;; Look for shapes that are inside a component copy, but are
           ;; not the root. In this case, they must not be deleted,
           ;; but hidden (to be able to recover them more easily).
           ;; Unless we are doing a component swap, in which case we want
           ;; to delete the old shape
-          (let [shape           (get objects shape-id)]
-            (and (ctn/has-any-copy-parent? objects shape)
-                 (not component-swap))))
+           (let [shape           (get objects shape-id)]
+             (and (ctn/has-any-copy-parent? objects shape)
+                  (not component-swap))))
 
-        [ids-to-delete ids-to-hide]
-        (if components-v2
-          (loop [ids-seq       (seq ids)
-                 ids-to-delete []
-                 ids-to-hide   []]
-            (let [id (first ids-seq)]
-              (if (nil? id)
-                [ids-to-delete ids-to-hide]
-                (if (in-component-copy? id)
-                  (recur (rest ids-seq)
-                         ids-to-delete
-                         (conj ids-to-hide id))
-                  (recur (rest ids-seq)
-                         (conj ids-to-delete id)
-                         ids-to-hide)))))
-          [ids []])
+         [ids-to-delete ids-to-hide]
+         (loop [ids-seq       (seq ids)
+                ids-to-delete []
+                ids-to-hide   []]
+           (let [id (first ids-seq)]
+             (if (nil? id)
+               [ids-to-delete ids-to-hide]
+               (if (in-component-copy? id)
+                 (recur (rest ids-seq)
+                        ids-to-delete
+                        (conj ids-to-hide id))
+                 (recur (rest ids-seq)
+                        (conj ids-to-delete id)
+                        ids-to-hide)))))
 
-        changes (-> changes
-                    (pcb/with-page page)
-                    (pcb/with-objects objects)
-                    (pcb/with-library-data file))
+         lookup  (d/getf objects)
 
-        lookup  (d/getf objects)
-
-        groups-to-unmask
-        (reduce (fn [group-ids id]
+         groups-to-unmask
+         (reduce (fn [group-ids id]
                   ;; When the shape to delete is the mask of a masked group,
                   ;; the mask condition must be removed, and it must be
                   ;; converted to a normal group.
-                  (let [obj    (lookup id)
-                        parent (lookup (:parent-id obj))]
-                    (if (and (:masked-group parent)
-                             (= id (first (:shapes parent))))
-                      (conj group-ids (:id parent))
-                      group-ids)))
-                #{}
-                ids-to-delete)
+                   (let [obj    (lookup id)
+                         parent (lookup (:parent-id obj))]
+                     (if (and (:masked-group parent)
+                              (= id (first (:shapes parent))))
+                       (conj group-ids (:id parent))
+                       group-ids)))
+                 #{}
+                 ids-to-delete)
 
-        interacting-shapes
-        (filter (fn [shape]
+         interacting-shapes
+         (filter (fn [shape]
                   ;; If any of the deleted shapes is the destination of
                   ;; some interaction, this must be deleted, too.
-                  (let [interactions (:interactions shape)]
-                    (some #(and (ctsi/has-destination %)
-                                (contains? ids-to-delete (:destination %)))
-                          interactions)))
-                (vals objects))
+                   (let [interactions (:interactions shape)]
+                     (some #(and (ctsi/has-destination %)
+                                 (contains? ids-to-delete (:destination %)))
+                           interactions)))
+                 (vals objects))
 
-        changes
-        (reduce (fn [changes {:keys [id] :as flow}]
-                  (if (contains? ids-to-delete (:starting-frame flow))
-                    (pcb/set-flow changes id nil)
-                    changes))
-                changes
-                (:flows page))
+         changes
+         (reduce (fn [changes {:keys [id] :as flow}]
+                   (if (contains? ids-to-delete (:starting-frame flow))
+                     (-> changes
+                         (pcb/with-page page)
+                         (pcb/set-flow id nil))
+                     changes))
+                 changes
+                 (:flows page))
 
 
-        all-parents
-        (reduce (fn [res id]
+         all-parents
+         (reduce (fn [res id]
                   ;; All parents of any deleted shape must be resized.
-                  (into res (cfh/get-parent-ids objects id)))
-                (d/ordered-set)
-                (concat ids-to-delete ids-to-hide))
+                   (into res (cfh/get-parent-ids objects id)))
+                 (d/ordered-set)
+                 (concat ids-to-delete ids-to-hide))
 
-        all-children
-        (->> ids-to-delete ;; Children of deleted shapes must be also deleted.
-             (reduce (fn [res id]
-                       (into res (cfh/get-children-ids objects id)))
-                     [])
-             (reverse)
-             (into (d/ordered-set)))
+         all-children
+         (->> ids-to-delete ;; Children of deleted shapes must be also deleted.
+              (reduce (fn [res id]
+                        (into res (cfh/get-children-ids objects id)))
+                      [])
+              (reverse)
+              (into (d/ordered-set)))
 
-        find-all-empty-parents
-        (fn recursive-find-empty-parents [empty-parents]
-          (let [all-ids   (into empty-parents ids-to-delete)
-                contains? (partial contains? all-ids)
-                xform     (comp (map lookup)
-                                (filter #(or (cfh/group-shape? %) (cfh/bool-shape? %)))
-                                (remove #(->> (:shapes %) (remove contains?) seq))
-                                (map :id))
-                parents   (into #{} xform all-parents)]
-            (if (= empty-parents parents)
-              empty-parents
-              (recursive-find-empty-parents parents))))
+         find-all-empty-parents
+         (fn recursive-find-empty-parents [empty-parents]
+           (let [all-ids   (into empty-parents ids-to-delete)
+                 contains? (partial contains? all-ids)
+                 xform     (comp (map lookup)
+                                 (filter #(or (cfh/group-shape? %) (cfh/bool-shape? %) (ctk/is-variant-container? %)))
+                                 (remove #(->> (:shapes %) (remove contains?) seq))
+                                 (map :id))
+                 parents   (into #{} xform all-parents)]
+             (if (= empty-parents parents)
+               empty-parents
+               (recursive-find-empty-parents parents))))
 
-        empty-parents
+         empty-parents
         ;; Any parent whose children are all deleted, must be deleted too.
         ;; Unless we are during a component swap: in this case we are replacing a shape by
         ;; other one, so must not delete empty parents.
-        (if-not component-swap
-          (into (d/ordered-set) (find-all-empty-parents #{}))
-          #{})
+         (if-not component-swap
+           (into (d/ordered-set) (find-all-empty-parents #{}))
+           #{})
 
-        components-to-delete
-        (if components-v2
-          (reduce (fn [components id]
-                    (let [shape (get objects id)]
-                      (if (and (= (:component-file shape) (:id file)) ;; Main instances should exist only in local file
-                               (:main-instance shape))                ;; but check anyway
-                        (conj components (:component-id shape))
-                        components)))
-                  []
-                  (into ids-to-delete all-children))
-          [])
+         components-to-delete
+         (reduce (fn [components id]
+                   (let [shape (get objects id)]
+                     (if (and (= (:component-file shape) (:id data)) ;; Main instances should exist only in local file
+                              (:main-instance shape))                ;; but check anyway
+                       (conj components (:component-id shape))
+                       components)))
+                 []
+                 (into ids-to-delete all-children))
 
-        ids-set (set ids-to-delete)
 
-        guides-to-delete
-        (->> (:guides page)
-             (vals)
-             (filter #(contains? ids-set (:frame-id %)))
-             (map :id))
+         ids-set (set ids-to-delete)
 
-        changes (reduce (fn [changes guide-id]
-                          (pcb/set-flow changes guide-id nil))
-                        changes
-                        guides-to-delete)
+         guides-to-delete
+         (->> (:guides page)
+              (vals)
+              (filter #(contains? ids-set (:frame-id %)))
+              (map :id))
 
-        changes (reduce (fn [changes component-id]
+         changes (reduce (fn [changes guide-id]
+                           (-> changes
+                               (pcb/with-page page)
+                               (pcb/set-flow guide-id nil)))
+                         changes
+                         guides-to-delete)
+
+         changes (reduce (fn [changes component-id]
                           ;; It's important to delete the component before the main instance, because we
                           ;; need to store the instance position if we want to restore it later.
-                          (pcb/delete-component changes component-id (:id page)))
-                        changes
-                        components-to-delete)
+                           (pcb/delete-component changes component-id (:id page)))
+                         changes
+                         components-to-delete)
 
-        changes (-> changes
-                    (generate-update-shape-flags ids-to-hide objects {:hidden true})
-                    (pcb/remove-objects all-children {:ignore-touched true})
-                    (pcb/remove-objects ids-to-delete {:ignore-touched ignore-touched})
-                    (pcb/remove-objects empty-parents)
-                    (pcb/resize-parents all-parents)
-                    (pcb/update-shapes groups-to-unmask
-                                       (fn [shape]
-                                         (assoc shape :masked-group false)))
-                    (pcb/update-shapes (map :id interacting-shapes)
-                                       (fn [shape]
-                                         (d/update-when shape :interactions
-                                                        (fn [interactions]
-                                                          (into []
-                                                                (remove #(and (ctsi/has-destination %)
-                                                                              (contains? ids-to-delete (:destination %))))
-                                                                interactions))))))]
-    [all-parents changes]))
+         changes (-> changes
+                     (generate-update-shape-flags ids-to-hide objects {:hidden true})
+                     (pcb/remove-objects all-children {:ignore-touched true})
+                     (pcb/remove-objects ids-to-delete {:ignore-touched ignore-touched})
+                     (pcb/remove-objects empty-parents)
+                     (pcb/resize-parents all-parents)
+                     (pcb/update-shapes groups-to-unmask
+                                        (fn [shape]
+                                          (assoc shape :masked-group false)))
+                     (pcb/update-shapes (map :id interacting-shapes)
+                                        (fn [shape]
+                                          (d/update-when shape :interactions
+                                                         (fn [interactions]
+                                                           (into []
+                                                                 (remove #(and (ctsi/has-destination %)
+                                                                               (contains? ids-to-delete (:destination %))))
+                                                                 interactions))))))]
+     [all-parents changes])))
 
 
 (defn generate-relocate
-  [changes objects parent-id page-id to-index ids & {:keys [cell ignore-parents?]}]
-  (let [ids    (cfh/order-by-indexed-shapes objects ids)
-        shapes (map (d/getf objects) ids)
-        parent (get objects parent-id)
+  [changes parent-id to-index ids & {:keys [cell ignore-parents?]}]
+  (let [objects     (pcb/get-objects changes)
+        ids         (cfh/order-by-indexed-shapes objects ids)
+        shapes      (map (d/getf objects) ids)
+        parent      (get objects parent-id)
         all-parents (into #{parent-id} (map #(cfh/get-parent-id objects %)) ids)
-        parents  (if ignore-parents? #{parent-id} all-parents)
+        parents     (if ignore-parents? #{parent-id} all-parents)
 
-        children-ids
-        (->> ids
-             (mapcat #(cfh/get-children-ids-with-self objects %)))
+        children-ids (mapcat #(cfh/get-children-ids-with-self objects %) ids)
 
-        child-heads
-        (->> ids
-             (mapcat #(ctn/get-child-heads objects %))
-             (map :id))
+        child-heads (mapcat #(ctn/get-child-heads objects %) ids)
+
+        child-heads-ids (map :id child-heads)
+
+        variant-shapes (filter ctk/is-variant? shapes)
 
         component-main-parent
         (ctn/find-component-main objects parent false)
@@ -337,12 +347,21 @@
                      (map :id)))
 
         index-cell-data  (when to-index (ctl/get-cell-by-index parent to-index))
-        cell (or cell (and index-cell-data [(:row index-cell-data) (:column index-cell-data)]))]
+        cell (or cell (and index-cell-data [(:row index-cell-data) (:column index-cell-data)]))
+
+
+        ;; Parents that are a variant-container that becomes empty
+        empty-variant-cont (reduce
+                            (fn [to-delete parent-id]
+                              (let [parent (get objects parent-id)]
+                                (if (and (ctk/is-variant-container? parent)
+                                         (empty? (remove (set ids) (:shapes parent))))
+                                  (conj to-delete (:id parent))
+                                  to-delete)))
+                            #{}
+                            (remove #(= % parent-id) all-parents))]
 
     (-> changes
-        (pcb/with-page-id page-id)
-        (pcb/with-objects objects)
-
         ;; Remove layout-item properties when moving a shape outside a layout
         (cond-> (not (ctl/any-layout? parent))
           (pcb/update-shapes ids ctl/remove-layout-item-data))
@@ -353,7 +372,7 @@
 
         ;; Remove the swap slots if it is moving to a different component
         (pcb/update-shapes
-         child-heads
+         child-heads-ids
          (fn [shape]
            (cond-> shape
              (not= component-main-parent (ctn/find-component-main objects shape false))
@@ -365,7 +384,15 @@
 
         ;; Add component-root property when moving a component outside a component
         (cond-> (not (ctn/get-instance-root objects parent))
-          (pcb/update-shapes child-heads #(assoc % :component-root true)))
+          (pcb/update-shapes child-heads-ids #(assoc % :component-root true)))
+
+        ;; Remove variant info and rename when moving outside a variant-container
+        (cond-> (not (ctk/is-variant-container? parent))
+          (clvp/generate-make-shapes-no-variant variant-shapes))
+
+        ;; Add variant info and rename when moving into a different variant-container
+        (cond-> (ctk/is-variant-container? parent)
+          (clvp/generate-make-shapes-variant child-heads parent))
 
         ;; Move the shapes
         (pcb/change-parent parent-id
@@ -440,7 +467,11 @@
           (pcb/update-shapes ids #(assoc % :blocked true)))
 
         ;; Resize parent containers that need to
-        (pcb/resize-parents parents))))
+        (pcb/resize-parents parents)
+
+        ;; Remove parents when are a variant-container that becomes empty
+        (cond-> (seq empty-variant-cont)
+          (#(second (generate-delete-shapes % empty-variant-cont {})))))))
 
 (defn change-show-in-viewer
   [shape hide?]