📎 Start new development cycle for 2.5

.circleci/config.yml

@@ -33,12 +33,6 @@ jobs:
           command: |
             clojure -M:dev:test
 
-      - run:
-          name: "NODE tests"
-          working_directory: "./common"
-          command: |
-            yarn run test
-
       - save_cache:
           paths:
             - ~/.m2

CHANGES.md

@@ -1,32 +1,17 @@
 # CHANGELOG
 
-## 2.4.3
+## 2.5.0
+
+### :rocket: Epics and highlights
+
+### :boom: Breaking changes & Deprecations
+
+### :heart: Community contributions (Thank you!)
+
+### :sparkles: New features
 
 ### :bug: Bugs fixed
 
-- Fix errors from editable select on measures menu [Taiga #9888](https://tree.taiga.io/project/penpot/issue/9888)
-- Fix exception on importing some templates from templates slider
-- Consolidate adding share button to workspace
-- Fix problem when pasting text [Taiga #9929](https://tree.taiga.io/project/penpot/issue/9929)
-- Fix incorrect media reference handling on component instantiation
-
-
-## 2.4.2
-
-### :bug: Bugs fixed
-
-- Fix detach when top copy is dangling and nested copy is not [Taiga #9699](https://tree.taiga.io/project/penpot/issue/9699)
-- Fix problem in plugins with `replaceColor` method [#174](https://github.com/penpot/penpot-plugins/issues/174)
-- Fix issue with recursive commponents [Taiga #9903](https://tree.taiga.io/project/penpot/issue/9903)
-- Fix missing methods reference on API Docs
-- Fix memory usage issue on file-gc asynchronous task (related to snapshots feature)
-
-## 2.4.1
-
-### :bug: Bugs fixed
-
-- Fix error when importing files with touched components [Taiga #9625](https://tree.taiga.io/project/penpot/issue/9625)
-- Fix problem when changing color libraries [Plugins #184](https://github.com/penpot/penpot-plugins/issues/184)
 
 ## 2.4.0
 
@@ -39,36 +24,30 @@
   (penpot). Because of that, the default NGINX listen port is now 8080 instead of 80, so
   you will have to modify your infrastructure to apply this change.
 
-- Redis 7.2 is explicitly pinned in our example docker-compose.yml file. This is done because,
-  starting with the next versions, Redis is no longer distributed under an open-source license.
-  On-premise users are obviously free to upgrade to the version they are using or a more modern one.
-  Keep in mind that if you were using a version other than 7.2, you may have to recreate the volume
-  associated with the Redis container because the 7.2 storage format may not be compatible with what
-  you already have stored on the volume, and Redis may not start. In the near future, we will evaluate
+- Redis 7.2 is explicitly pinned in our example docker-compose.yml file. This is done because, 
+  starting with the next versions, Redis is no longer distributed under an open-source license. 
+  On-premise users are obviously free to upgrade to the version they are using or a more modern one. 
+  Keep in mind that if you were using a version other than 7.2, you may have to recreate the volume 
+  associated with the Redis container because the 7.2 storage format may not be compatible with what 
+  you already have stored on the volume, and Redis may not start. In the near future, we will evaluate 
   whether to move to an open-source version of Redis (such as https://valkey.io/).
 
 ### :heart: Community contributions (Thank you!)
 
 ### :sparkles: New features
 
-- Viewer role for team members [Taiga #1056](https://tree.taiga.io/project/penpot/us/1056) & [Taiga #6590](https://tree.taiga.io/project/penpot/us/6590)
-- File history versions management [Taiga #187](https://tree.taiga.io/project/penpot/us/187?milestone=411120)
+- Viewer role for team members  [Taiga #1056 & #6590](https://tree.taiga.io/project/penpot/us/1056 & https://tree.taiga.io/project/penpot/us/6590)
+- File history versions management [Taiga](https://tree.taiga.io/project/penpot/us/187?milestone=411120)
 - Rename selected layer via keyboard shortcut and context menu option [Taiga #8882](https://tree.taiga.io/project/penpot/us/8882)
-- New .penpot file format [Taiga #8657](https://tree.taiga.io/project/penpot/us/8657)
 
 ### :bug: Bugs fixed
 
-- Fix problem with some texts desynchronization [Taiga #9379](https://tree.taiga.io/project/penpot/issue/9379)
-- Fix problem with reoder grid layers [#5446](https://github.com/penpot/penpot/issues/5446)
-- Fix problem with swap component style [#9542](https://tree.taiga.io/project/penpot/issue/9542)
 
 ## 2.3.3
 
 ### :bug: Bugs fixed
 
 - Fix problem creating manual overlay interactions [Taiga #9146](https://tree.taiga.io/project/penpot/issue/9146)
-- Fix plugins list default URL
-- Activate plugins feature by default
 
 ## 2.3.2
 

backend/resources/app/templates/debug.tmpl

@@ -114,13 +114,37 @@ Debug Main Page
     </fieldset>
     <fieldset>
       <legend>Import binfile:</legend>
-      <desc>Import penpot file in binary format.</desc>
+      <desc>Import penpot file in binary
+      format. If <strong>overwrite</strong> is checked, all files will
+      be overwritten using the same ids found in the file instead of
+      generating a new ones.</desc>
 
       <form method="post" enctype="multipart/form-data" action="/dbg/file/import">
         <div class="row">
           <input type="file" name="file" value="" />
         </div>
 
+        <div class="row">
+          <label>Overwrite?</label>
+          <input type="checkbox" name="overwrite" />
+          <br />
+          <small>
+            Instead of creating a new file with all relations remapped,
+            reuses all ids and updates/overwrites the objects that are
+            already exists on the database.
+            <strong>Warning, this operation should be used with caution.</strong>
+          </small>
+        </div>
+
+        <div class="row">
+          <label>Migrate?</label>
+          <input type="checkbox" name="migrate" />
+          <br />
+          <small>
+            Applies the file migrations on the importation process.
+          </small>
+        </div>
+
         <div class="row">
           <input type="submit" name="upload" value="Upload" />
         </div>

backend/scripts/repl

@@ -1,6 +1,5 @@
 #!/usr/bin/env bash
 
-export PENPOT_SECRET_KEY=super-secret-devenv-key
 export PENPOT_HOST=devenv
 export PENPOT_FLAGS="\
        $PENPOT_FLAGS \

backend/scripts/start-dev

@@ -1,6 +1,5 @@
 #!/usr/bin/env bash
 
-export PENPOT_SECRET_KEY=super-secret-devenv-key
 export PENPOT_HOST=devenv
 export PENPOT_FLAGS="\
        $PENPOT_FLAGS \

backend/src/app/auth.clj

@@ -8,7 +8,7 @@
   (:require
    [buddy.hashers :as hashers]))
 
-(def ^:private default-options
+(def default-params
   {:alg :argon2id
    :memory 32768 ;; 32 MiB
    :iterations 3
@@ -16,12 +16,12 @@
 
 (defn derive-password
   [password]
-  (hashers/derive password default-options))
+  (hashers/derive password default-params))
 
 (defn verify-password
   [attempt password]
   (try
-    (hashers/verify attempt password default-options)
+    (hashers/verify attempt password)
     (catch Throwable _
       {:update false
        :valid false})))

backend/src/app/binfile/common.clj

@@ -12,7 +12,6 @@
    [app.common.data.macros :as dm]
    [app.common.exceptions :as ex]
    [app.common.features :as cfeat]
-   [app.common.files.helpers :as cfh]
    [app.common.files.migrations :as fmg]
    [app.common.files.validate :as fval]
    [app.common.logging :as l]
@@ -30,9 +29,8 @@
    [app.util.time :as dt]
    [app.worker :as-alias wrk]
    [clojure.set :as set]
-   [cuerdas.core :as str]
-   [datoteka.fs :as fs]
-   [datoteka.io :as io]))
+   [clojure.walk :as walk]
+   [cuerdas.core :as str]))
 
 (set! *warn-on-reflection* true)
 
@@ -54,20 +52,6 @@
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
-(defn parse-file-format
-  [template]
-  (assert (fs/path? template) "expected InputStream for `template`")
-
-  (with-open [^java.lang.AutoCloseable input (io/input-stream template)]
-    (let [buffer (byte-array 4)]
-      (io/read-to-buffer input buffer)
-      (if (and (= (aget buffer 0) 80)
-               (= (aget buffer 1) 75)
-               (= (aget buffer 2) 3)
-               (= (aget buffer 3) 4))
-        :binfile-v3
-        :binfile-v1))))
-
 (def xf-map-id
   (map :id))
 
@@ -150,16 +134,6 @@
                          (update :data feat.fdata/process-pointers deref)
                          (update :data feat.fdata/process-objects (partial into {}))))))))
 
-(defn clean-file-features
-  [file]
-  (update file :features (fn [features]
-                           (if (set? features)
-                             (-> features
-                                 (cfeat/migrate-legacy-features)
-                                 (set/difference cfeat/frontend-only-features)
-                                 (set/difference cfeat/backend-only-features))
-                             #{}))))
-
 (defn get-project
   [cfg project-id]
   (db/get cfg :project {:id project-id}))
@@ -241,65 +215,40 @@
             :data nil}
            {::sql/columns [:media-id :file-id :revn]}))
 
-(def ^:private sql:get-missing-media-references
-  "SELECT fmo.*
-     FROM file_media_object AS fmo
-    WHERE fmo.id = ANY(?::uuid[])
-      AND file_id != ?")
 
-(defn update-media-references!
-  "Given a file and a coll of media-refs, check if all provided
-  references are correct or fix them in-place"
-  [{:keys [::db/conn] :as cfg} {file-id :id :as file} media-refs]
-  (let [missing-index
-        (reduce (fn [result {:keys [id] :as fmo}]
-                  (assoc result id
-                         (-> fmo
-                             (assoc :id (uuid/next))
-                             (assoc :file-id file-id)
-                             (dissoc :created-at)
-                             (dissoc :deleted-at))))
-                {}
-                (db/exec! conn [sql:get-missing-media-references
-                                (->> (into #{} xf-map-id media-refs)
-                                     (db/create-array conn "uuid"))
-                                file-id]))
+(def ^:private
+  xform:collect-media-id
+  (comp
+   (map :objects)
+   (mapcat vals)
+   (mapcat (fn [obj]
+             ;; NOTE: because of some bug, we ended with
+             ;; many shape types having the ability to
+             ;; have fill-image attribute (which initially
+             ;; designed for :path shapes).
+             (sequence
+              (keep :id)
+              (concat [(:fill-image obj)
+                       (:metadata obj)]
+                      (map :fill-image (:fills obj))
+                      (map :stroke-image (:strokes obj))
+                      (->> (:content obj)
+                           (tree-seq map? :children)
+                           (mapcat :fills)
+                           (map :fill-image))))))))
 
-        lookup-index
-        (fn [id]
-          (if-let [mobj (get missing-index id)]
-            (do
-              (l/trc :hint "lookup index"
-                     :file-id (str file-id)
-                     :snap-id (str (:snapshot-id file))
-                     :id (str id)
-                     :result (str (get mobj :id)))
-              (get mobj :id))
-
-            id))
-
-        update-shapes
-        (fn [data {:keys [page-id shape-id]}]
-          (d/update-in-when data [:pages-index page-id :objects shape-id] cfh/relink-media-refs lookup-index))
-
-        file
-        (update file :data #(reduce update-shapes % media-refs))]
-
-    (doseq [[old-id item] missing-index]
-      (l/dbg :hint "create missing references"
-             :file-id (str file-id)
-             :snap-id (str (:snapshot-id file))
-             :old-id (str old-id)
-             :id (str (:id item)))
-      (db/insert! conn :file-media-object item
-                  {::db/return-keys false}))
-
-    file))
+(defn collect-used-media
+  "Given a fdata (file data), returns all media references."
+  [data]
+  (-> #{}
+      (into xform:collect-media-id (vals (:pages-index data)))
+      (into xform:collect-media-id (vals (:components data)))
+      (into (keys (:media data)))))
 
 (defn get-file-media
   [cfg {:keys [data id] :as file}]
   (db/run! cfg (fn [{:keys [::db/conn]}]
-                 (let [ids (cfh/collect-used-media data)
+                 (let [ids (collect-used-media data)
                        ids (db/create-array conn "uuid" ids)
                        sql (str "SELECT * FROM file_media_object WHERE id = ANY(?)")]
 
@@ -352,7 +301,48 @@
   replace the old :component-file reference with the new
   ones, using the provided file-index."
   [data]
-  (cfh/relink-media-refs data lookup-index))
+  (letfn [(process-map-form [form]
+            (cond-> form
+              ;; Relink image shapes
+              (and (map? (:metadata form))
+                   (= :image (:type form)))
+              (update-in [:metadata :id] lookup-index)
+
+              ;; Relink paths with fill image
+              (map? (:fill-image form))
+              (update-in [:fill-image :id] lookup-index)
+
+              ;; This covers old shapes and the new :fills.
+              (uuid? (:fill-color-ref-file form))
+              (update :fill-color-ref-file lookup-index)
+
+              ;; This covers the old shapes and the new :strokes
+              (uuid? (:stroke-color-ref-file form))
+              (update :stroke-color-ref-file lookup-index)
+
+              ;; This covers all text shapes that have typography referenced
+              (uuid? (:typography-ref-file form))
+              (update :typography-ref-file lookup-index)
+
+              ;; This covers the component instance links
+              (uuid? (:component-file form))
+              (update :component-file lookup-index)
+
+              ;; This covers the shadows and grids (they have directly
+              ;; the :file-id prop)
+              (uuid? (:file-id form))
+              (update :file-id lookup-index)))
+
+          (process-form [form]
+            (if (map? form)
+              (try
+                (process-map-form form)
+                (catch Throwable cause
+                  (l/warn :hint "failed form" :form (pr-str form) ::l/sync? true)
+                  (throw cause)))
+              form))]
+
+    (walk/postwalk process-form data)))
 
 (defn- relink-media
   "A function responsible of process the :media attr of file data and
@@ -455,11 +445,8 @@
                            (fn [features]
                              (let [features (cfeat/check-supported-features! features)]
                                (-> (::features cfg #{})
-                                   (set/union features)
-                                   ;; We never want to store
-                                   ;; frontend-only features on file
-                                   (set/difference cfeat/frontend-only-features))))))
-
+                                   (set/difference cfeat/frontend-only-features)
+                                   (set/union features))))))
 
         _      (when (contains? cf/flags :file-schema-validation)
                  (fval/validate-file-schema! file))

backend/src/app/binfile/v1.clj

@@ -298,7 +298,7 @@
 (defmulti write-section ::section)
 
 (defn write-export!
-  [{:keys [::bfc/include-libraries ::bfc/embed-assets] :as cfg}]
+  [{:keys [::include-libraries ::embed-assets] :as cfg}]
   (when (and include-libraries embed-assets)
     (throw (IllegalArgumentException.
             "the `include-libraries` and `embed-assets` are mutally excluding options")))
@@ -323,7 +323,7 @@
             [:v1/metadata :v1/files :v1/rels :v1/sobjects]))))
 
 (defmethod write-section :v1/metadata
-  [{:keys [::output ::bfc/ids ::bfc/include-libraries] :as cfg}]
+  [{:keys [::output ::ids ::include-libraries] :as cfg}]
   (if-let [fids (get-files cfg ids)]
     (let [lids (when include-libraries
                  (bfc/get-libraries cfg ids))
@@ -335,7 +335,7 @@
               :hint "unable to retrieve files for export")))
 
 (defmethod write-section :v1/files
-  [{:keys [::output ::bfc/embed-assets ::bfc/include-libraries] :as cfg}]
+  [{:keys [::output ::embed-assets ::include-libraries] :as cfg}]
 
   ;; Initialize SIDS with empty vector
   (vswap! bfc/*state* assoc :sids [])
@@ -382,7 +382,7 @@
       (vswap! bfc/*state* update :sids into bfc/xf-map-media-id thumbnails))))
 
 (defmethod write-section :v1/rels
-  [{:keys [::output ::bfc/include-libraries] :as cfg}]
+  [{:keys [::output ::include-libraries] :as cfg}]
   (let [ids  (-> bfc/*state* deref :files set)
         rels (when include-libraries
                (bfc/get-files-rels cfg ids))]
@@ -421,15 +421,15 @@
 (defmulti read-import ::version)
 (defmulti read-section ::section)
 
-(s/def ::bfc/profile-id ::us/uuid)
-(s/def ::bfc/project-id ::us/uuid)
-(s/def ::bfc/input io/input-stream?)
+(s/def ::profile-id ::us/uuid)
+(s/def ::project-id ::us/uuid)
+(s/def ::input io/input-stream?)
 (s/def ::overwrite? (s/nilable ::us/boolean))
 (s/def ::ignore-index-errors? (s/nilable ::us/boolean))
 
 ;; FIXME: replace with schema
 (s/def ::read-import-options
-  (s/keys :req [::db/pool ::sto/storage ::bfc/project-id ::bfc/profile-id ::bfc/input]
+  (s/keys :req [::db/pool ::sto/storage ::project-id ::profile-id ::input]
           :opt [::overwrite? ::ignore-index-errors?]))
 
 (defn read-import!
@@ -439,7 +439,7 @@
 
   `::bfc/overwrite`: if true, instead of creating new files and remapping id references,
   it reuses all ids and updates existing objects; defaults to `false`."
-  [{:keys [::bfc/input ::bfc/timestamp] :or {timestamp (dt/now)} :as options}]
+  [{:keys [::input ::bfc/timestamp] :or {timestamp (dt/now)} :as options}]
 
   (dm/assert!
    "expected input stream"
@@ -453,7 +453,7 @@
     (read-import (assoc options ::version version ::bfc/timestamp timestamp))))
 
 (defn- read-import-v1
-  [{:keys [::db/conn ::bfc/project-id ::bfc/profile-id ::bfc/input] :as cfg}]
+  [{:keys [::db/conn ::project-id ::profile-id ::input] :as cfg}]
 
   (bfc/disable-database-timeouts! cfg)
 
@@ -473,7 +473,7 @@
                 (let [options (-> cfg
                                   (assoc ::bfc/features features)
                                   (assoc ::section section)
-                                  (assoc ::bfc/input input))]
+                                  (assoc ::input input))]
                   (binding [bfc/*options* options]
                     (events/tap :progress {:op :import :section section})
                     (read-section options))))
@@ -491,7 +491,7 @@
   (db/tx-run! options read-import-v1))
 
 (defmethod read-section :v1/metadata
-  [{:keys [::bfc/input]}]
+  [{:keys [::input]}]
   (let [{:keys [version files]} (read-obj! input)]
     (l/dbg :hint "metadata readed"
            :version (:full version)
@@ -508,8 +508,17 @@
               (update :object-id #(str/replace-first % #"^(.*?)/" (str file-id "/")))))
         thumbnails))
 
+(defn- clean-features
+  [file]
+  (update file :features (fn [features]
+                           (if (set? features)
+                             (-> features
+                                 (cfeat/migrate-legacy-features)
+                                 (set/difference cfeat/backend-only-features))
+                             #{}))))
+
 (defmethod read-section :v1/files
-  [{:keys [::db/conn ::bfc/input ::bfc/project-id ::bfc/overwrite ::bfc/name] :as system}]
+  [{:keys [::db/conn ::input ::project-id ::bfc/overwrite ::name] :as system}]
 
   (doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))]
     (let [file       (read-obj! input)
@@ -518,7 +527,7 @@
           file-id    (:id file)
           file-id'   (bfc/lookup-index file-id)
 
-          file       (bfc/clean-file-features file)
+          file       (clean-features file)
           thumbnails (:thumbnails file)]
 
       (when (not= file-id expected-file-id)
@@ -576,7 +585,7 @@
         file-id'))))
 
 (defmethod read-section :v1/rels
-  [{:keys [::db/conn ::bfc/input ::bfc/timestamp]}]
+  [{:keys [::db/conn ::input ::bfc/timestamp]}]
   (let [rels (read-obj! input)
         ids  (into #{} (-> bfc/*state* deref :files))]
     ;; Insert all file relations
@@ -600,7 +609,7 @@
                   ::l/sync? true))))))
 
 (defmethod read-section :v1/sobjects
-  [{:keys [::db/conn ::bfc/input ::bfc/overwrite ::bfc/timestamp] :as cfg}]
+  [{:keys [::db/conn ::input ::bfc/overwrite ::bfc/timestamp] :as cfg}]
   (let [storage (sto/resolve cfg)
         ids     (read-obj! input)
         thumb?  (into #{} (map :media-id) (:thumbnails @bfc/*state*))]
@@ -674,17 +683,17 @@
   "Do the exportation of a specified file in custom penpot binary
   format. There are some options available for customize the output:
 
-  `::bfc/include-libraries`: additionally to the specified file, all the
+  `::include-libraries`: additionally to the specified file, all the
   linked libraries also will be included (including transitive
   dependencies).
 
-  `::bfc/embed-assets`: instead of including the libraries, embed in the
+  `::embed-assets`: instead of including the libraries, embed in the
   same file library all assets used from external libraries."
 
-  [{:keys [::bfc/ids] :as cfg} output]
+  [{:keys [::ids] :as cfg} output]
 
   (dm/assert!
-   "expected a set of uuid's for `::bfc/ids` parameter"
+   "expected a set of uuid's for `::ids` parameter"
    (and (set? ids)
         (every? uuid? ids)))
 
@@ -719,12 +728,12 @@
                 :cause @cs)))))
 
 (defn import-files!
-  [{:keys [::bfc/input] :as cfg}]
+  [{:keys [::input] :as cfg}]
 
   (dm/assert!
    "expected valid profile-id and project-id on `cfg`"
-   (and (uuid? (::bfc/profile-id cfg))
-        (uuid? (::bfc/project-id cfg))))
+   (and (uuid? (::profile-id cfg))
+        (uuid? (::project-id cfg))))
 
   (dm/assert!
    "expected instance of jio/IOFactory for `input`"
@@ -738,7 +747,7 @@
     (try
       (binding [*position* (atom 0)]
         (pu/with-open [input (io/input-stream input)]
-          (read-import! (assoc cfg ::bfc/input input))))
+          (read-import! (assoc cfg ::input input))))
 
       (catch ZstdIOException cause
         (ex/raise :type :validation

backend/src/app/binfile/v3.clj

@@ -12,7 +12,6 @@
    [app.common.data :as d]
    [app.common.data.macros :as dm]
    [app.common.exceptions :as ex]
-   [app.common.features :as cfeat]
    [app.common.json :as json]
    [app.common.logging :as l]
    [app.common.schema :as sm]
@@ -56,8 +55,7 @@
      [:map
       [:id ::sm/uuid]
       [:name :string]
-      [:project-id ::sm/uuid]
-      [:features ::cfeat/features]]]]
+      [:project-id ::sm/uuid]]]]
 
    [:relations {:optional true}
     [:vector
@@ -192,7 +190,7 @@
   (.closeEntry output))
 
 (defn- get-file
-  [{:keys [::bfc/embed-assets ::bfc/include-libraries] :as cfg} file-id]
+  [{:keys [::embed-assets ::include-libraries] :as cfg} file-id]
 
   (when (and include-libraries embed-assets)
     (throw (IllegalArgumentException.
@@ -205,10 +203,7 @@
           (dissoc :libraries))
 
       embed-assets
-      (update :data #(bfc/embed-assets cfg % file-id))
-
-      :always
-      (bfc/clean-file-features))))
+      (update :data #(bfc/embed-assets cfg % file-id)))))
 
 (defn- resolve-extension
   [mtype]
@@ -264,8 +259,7 @@
     (vswap! bfc/*state* update :files assoc file-id
             {:id file-id
              :project-id (:project-id file)
-             :name (:name file)
-             :features (:features file)})
+             :name (:name file)})
 
     (let [file (cond-> (dissoc file :data)
                  (:options data)
@@ -330,7 +324,7 @@
         (write-entry! output path color)))))
 
 (defn- export-files
-  [{:keys [::bfc/ids ::bfc/include-libraries ::output] :as cfg}]
+  [{:keys [::ids ::include-libraries ::output] :as cfg}]
   (let [ids  (into ids (when include-libraries (bfc/get-libraries cfg ids)))
         rels (if include-libraries
                (->> (bfc/get-files-rels cfg ids)
@@ -509,7 +503,7 @@
     (json/read reader :key-fn json/read-kebab-key)))
 
 (defn- read-file
-  [{:keys [::bfc/input ::file-id]}]
+  [{:keys [::input ::file-id]}]
   (let [path  (str "files/" file-id ".json")
         entry (get-zip-entry input path)]
     (-> (read-entry input entry)
@@ -517,7 +511,7 @@
         (validate-file))))
 
 (defn- read-file-plugin-data
-  [{:keys [::bfc/input ::file-id]}]
+  [{:keys [::input ::file-id]}]
   (let [path  (str "files/" file-id "/plugin-data.json")
         entry (get-zip-entry* input path)]
     (some->> entry
@@ -526,7 +520,7 @@
              (validate-plugin-data))))
 
 (defn- read-file-media
-  [{:keys [::bfc/input ::file-id ::entries]}]
+  [{:keys [::input ::file-id ::entries]}]
   (->> (keep (match-media-entry-fn file-id) entries)
        (reduce (fn [result {:keys [id entry]}]
                  (let [object (->> (read-entry input entry)
@@ -540,7 +534,7 @@
        (not-empty)))
 
 (defn- read-file-colors
-  [{:keys [::bfc/input ::file-id ::entries]}]
+  [{:keys [::input ::file-id ::entries]}]
   (->> (keep (match-color-entry-fn file-id) entries)
        (reduce (fn [result {:keys [id entry]}]
                  (let [object (->> (read-entry input entry)
@@ -553,7 +547,7 @@
        (not-empty)))
 
 (defn- read-file-components
-  [{:keys [::bfc/input ::file-id ::entries]}]
+  [{:keys [::input ::file-id ::entries]}]
   (->> (keep (match-component-entry-fn file-id) entries)
        (reduce (fn [result {:keys [id entry]}]
                  (let [object (->> (read-entry input entry)
@@ -566,7 +560,7 @@
        (not-empty)))
 
 (defn- read-file-typographies
-  [{:keys [::bfc/input ::file-id ::entries]}]
+  [{:keys [::input ::file-id ::entries]}]
   (->> (keep (match-typography-entry-fn file-id) entries)
        (reduce (fn [result {:keys [id entry]}]
                  (let [object (->> (read-entry input entry)
@@ -579,7 +573,7 @@
        (not-empty)))
 
 (defn- read-file-shapes
-  [{:keys [::bfc/input ::file-id ::page-id ::entries] :as cfg}]
+  [{:keys [::input ::file-id ::page-id ::entries] :as cfg}]
   (->> (keep (match-shape-entry-fn file-id page-id) entries)
        (reduce (fn [result {:keys [id entry]}]
                  (let [object (->> (read-entry input entry)
@@ -592,7 +586,7 @@
        (not-empty)))
 
 (defn- read-file-pages
-  [{:keys [::bfc/input ::file-id ::entries] :as cfg}]
+  [{:keys [::input ::file-id ::entries] :as cfg}]
   (->> (keep (match-page-entry-fn file-id) entries)
        (keep (fn [{:keys [id entry]}]
                (let [page (->> (read-entry input entry)
@@ -608,7 +602,7 @@
                (d/ordered-map))))
 
 (defn- read-file-thumbnails
-  [{:keys [::bfc/input ::file-id ::entries] :as cfg}]
+  [{:keys [::input ::file-id ::entries] :as cfg}]
   (->> (keep (match-thumbnail-entry-fn file-id) entries)
        (reduce (fn [result {:keys [page-id frame-id tag entry]}]
                  (let [object (->> (read-entry input entry)
@@ -638,7 +632,7 @@
      :plugin-data plugin-data}))
 
 (defn- import-file
-  [{:keys [::db/conn ::bfc/project-id ::file-id ::file-name] :as cfg}]
+  [{:keys [::db/conn ::project-id ::file-id ::file-name] :as cfg}]
   (let [file-id'   (bfc/lookup-index file-id)
         file       (read-file cfg)
         media      (read-file-media cfg)
@@ -714,7 +708,7 @@
                      :library-file-id libr-id})))))
 
 (defn- import-storage-objects
-  [{:keys [::bfc/input ::entries ::bfc/timestamp] :as cfg}]
+  [{:keys [::input ::entries ::bfc/timestamp] :as cfg}]
   (events/tap :progress {:section :storage-objects})
 
   (let [storage (sto/resolve cfg)
@@ -810,7 +804,7 @@
                   {::db/on-conflict-do-nothing? (::bfc/overwrite cfg)}))))
 
 (defn- import-files
-  [{:keys [::bfc/timestamp ::bfc/input ::bfc/name] :or {timestamp (dt/now)} :as cfg}]
+  [{:keys [::bfc/timestamp ::input ::name] :or {timestamp (dt/now)} :as cfg}]
 
   (dm/assert!
    "expected zip file"
@@ -878,17 +872,17 @@
   "Do the exportation of a specified file in custom penpot binary
   format. There are some options available for customize the output:
 
-  `::bfc/include-libraries`: additionally to the specified file, all the
+  `::include-libraries`: additionally to the specified file, all the
   linked libraries also will be included (including transitive
   dependencies).
 
-  `::bfc/embed-assets`: instead of including the libraries, embed in the
+  `::embed-assets`: instead of including the libraries, embed in the
   same file library all assets used from external libraries."
 
-  [{:keys [::bfc/ids] :as cfg} output]
+  [{:keys [::ids] :as cfg} output]
 
   (dm/assert!
-   "expected a set of uuid's for `::bfc/ids` parameter"
+   "expected a set of uuid's for `::ids` parameter"
    (and (set? ids)
         (every? uuid? ids)))
 
@@ -930,13 +924,14 @@
                 :aborted @ab
                 :cause @cs)))))
 
+
 (defn import-files!
-  [{:keys [::bfc/input] :as cfg}]
+  [{:keys [::input] :as cfg}]
 
   (dm/assert!
    "expected valid profile-id and project-id on `cfg`"
-   (and (uuid? (::bfc/profile-id cfg))
-        (uuid? (::bfc/project-id cfg))))
+   (and (uuid? (::profile-id cfg))
+        (uuid? (::project-id cfg))))
 
   (dm/assert!
    "expected instance of jio/IOFactory for `input`"
@@ -949,7 +944,7 @@
     (l/info :hint "import: started" :id (str id))
     (try
       (with-open [input (ZipFile. (fs/file input))]
-        (import-files (assoc cfg ::bfc/input input)))
+        (import-files (assoc cfg ::input input)))
 
       (catch Throwable cause
         (vreset! cs cause)

backend/src/app/config.clj

@@ -42,6 +42,7 @@
    :rpc-rlimit-config "resources/rlimit.edn"
    :rpc-climit-config "resources/climit.edn"
 
+   :auto-file-snapshot-total 10
    :auto-file-snapshot-every 5
    :auto-file-snapshot-timeout "3h"
 
@@ -100,6 +101,7 @@
     [:telemetry-uri {:optional true} :string]
     [:telemetry-with-taiga {:optional true} ::sm/boolean] ;; DELETE
 
+    [:auto-file-snapshot-total {:optional true} ::sm/int]
     [:auto-file-snapshot-every {:optional true} ::sm/int]
     [:auto-file-snapshot-timeout {:optional true} ::dt/duration]
 
@@ -142,8 +144,6 @@
     [:quotes-comments-per-file {:optional true} ::sm/int]
     [:quotes-snapshots-per-file {:optional true} ::sm/int]
     [:quotes-snapshots-per-team {:optional true} ::sm/int]
-    [:quotes-team-access-requests-per-team {:optional true} ::sm/int]
-    [:quotes-team-access-requests-per-requester {:optional true} ::sm/int]
 
     [:auth-data-cookie-domain {:optional true} :string]
     [:auth-token-cookie-name {:optional true} :string]

backend/src/app/db.clj

@@ -270,17 +270,19 @@
     :else           (throw (IllegalArgumentException. "unable to resolve connectable"))))
 
 (def ^:private params-mapping
-  {::return-keys :return-keys})
+  {::return-keys? :return-keys
+   ::return-keys :return-keys})
 
 (defn rename-opts
   [opts]
   (set/rename-keys opts params-mapping))
 
 (def ^:private default-insert-opts
-  (assoc sql/default-opts :return-keys true))
+  {:builder-fn sql/as-kebab-maps
+   :return-keys true})
 
 (def ^:private default-opts
-  sql/default-opts)
+  {:builder-fn sql/as-kebab-maps})
 
 (defn exec!
   ([ds sv] (exec! ds sv nil))
@@ -331,7 +333,7 @@
 (defn update!
   "A helper that build an UPDATE SQL statement and executes it.
 
-  Given a connectable object, a table name, a hash map of columns and
+   Given a connectable object, a table name, a hash map of columns and
   values to set, and either a hash map of columns and values to search
   on or a vector of a SQL where clause and parameters, perform an
   update on the table.
@@ -411,20 +413,10 @@
                 :hint "database object not found"))
     row))
 
-(def ^:private default-plan-opts
-  (-> default-opts
-      (assoc :fetch-size 1)
-      (assoc :concurrency :read-only)
-      (assoc :cursors :close)
-      (assoc :result-type :forward-only)))
-
 (defn plan
-  ([ds sql]
-   (-> (get-connectable ds)
-       (jdbc/plan sql default-plan-opts)))
-  ([ds sql opts]
-   (-> (get-connectable ds)
-       (jdbc/plan sql (merge default-plan-opts opts)))))
+  [ds sql]
+  (-> (get-connectable ds)
+      (jdbc/plan sql sql/default-opts)))
 
 (defn cursor
   "Return a lazy seq of rows using server side cursors"

backend/src/app/db/sql.clj

@@ -15,15 +15,14 @@
 (defn kebab-case [s] (str/replace s #"_" "-"))
 (defn snake-case [s] (str/replace s #"-" "_"))
 
+(def default-opts
+  {:table-fn snake-case
+   :column-fn snake-case})
+
 (defn as-kebab-maps
   [rs opts]
   (jdbc-opt/as-unqualified-modified-maps rs (assoc opts :label-fn kebab-case)))
 
-(def default-opts
-  {:table-fn snake-case
-   :column-fn snake-case
-   :builder-fn as-kebab-maps})
-
 (defn insert
   ([table key-map]
    (insert table key-map nil))

backend/src/app/email.clj

@@ -226,8 +226,8 @@
    [:priority {:optional true} [:enum :high :low]]
    [:extra-data {:optional true} ::sm/text]])
 
-(def ^:private check-context
-  (sm/check-fn schema:context))
+(def ^:private valid-context?
+  (sm/validator schema:context))
 
 (defn template-factory
   [& {:keys [id schema]}]
@@ -236,8 +236,10 @@
                    (sm/check-fn schema)
                    (constantly nil))]
     (fn [context]
-      (let [context (-> context check-context check-fn)
-            email   (build-email-template id context)]
+      (assert (valid-context? context) "expected a valid context")
+      (check-fn context)
+
+      (let [email (build-email-template id context)]
         (when-not email
           (ex/raise :type :internal
                     :code :email-template-does-not-exists
@@ -269,7 +271,7 @@
   "Schedule an already defined email to be sent using asynchronously
   using worker task."
   [{:keys [::conn ::factory] :as context}]
-  (assert (db/connectable? conn) "expected a valid database connection or pool")
+  (assert (db/connection? conn) "expected a valid database connection")
 
   (let [email (if factory
                 (factory context)
@@ -346,7 +348,7 @@
    [:subject ::sm/text]
    [:content ::sm/text]])
 
-(def user-feedback
+(def feedback
   "A profile feedback email."
   (template-factory
    :id ::feedback

backend/src/app/http.clj

@@ -42,8 +42,8 @@
 (def default-params
   {::port 6060
    ::host "0.0.0.0"
-   ::max-body-size 31457280              ; default  30 MiB
-   ::max-multipart-body-size 367001600}) ; default 350 MiB
+   ::max-body-size (* 1024 1024 30)             ; default 30 MiB
+   ::max-multipart-body-size (* 1024 1024 120)}) ; default 120 MiB
 
 (defmethod ig/expand-key ::server
   [k v]

backend/src/app/http/debug.clj

@@ -7,9 +7,7 @@
 (ns app.http.debug
   (:refer-clojure :exclude [error-handler])
   (:require
-   [app.binfile.common :as bfc]
    [app.binfile.v1 :as bf.v1]
-   [app.binfile.v3 :as bf.v3]
    [app.common.data :as d]
    [app.common.exceptions :as ex]
    [app.common.logging :as l]
@@ -282,23 +280,23 @@
       (ex/raise :type :validation
                 :code :missing-arguments))
 
-    (let [path (tmp/tempfile :prefix "penpot.export." :min-age "30m")]
+    (let [path (tmp/tempfile :prefix "penpot.export.")]
       (with-open [output (io/output-stream path)]
         (-> cfg
-            (assoc ::bfc/ids file-ids)
-            (assoc ::bfc/embed-assets embed?)
-            (assoc ::bfc/include-libraries libs?)
-            (bf.v3/export-files! output)))
+            (assoc ::bf.v1/ids file-ids)
+            (assoc ::bf.v1/embed-assets embed?)
+            (assoc ::bf.v1/include-libraries libs?)
+            (bf.v1/export-files! output)))
 
       (if clone?
         (let [profile    (profile/get-profile pool profile-id)
               project-id (:default-project-id profile)
               cfg        (assoc cfg
-                                ::bfc/overwrite false
-                                ::bfc/profile-id profile-id
-                                ::bfc/project-id project-id
-                                ::bfc/input path)]
-          (bf.v3/import-files! cfg)
+                                ::bf.v1/overwrite false
+                                ::bf.v1/profile-id profile-id
+                                ::bf.v1/project-id project-id
+                                ::bf.v1/input path)]
+          (bf.v1/import-files! cfg)
           {::yres/status  200
            ::yres/headers {"content-type" "text/plain"}
            ::yres/body    "OK CLONED"})
@@ -317,24 +315,23 @@
               :hint "missing upload file"))
 
   (let [profile    (profile/get-profile pool profile-id)
-        project-id (:default-project-id profile)]
+        project-id (:default-project-id profile)
+        overwrite? (contains? params :overwrite)
+        migrate?   (contains? params :migrate)]
 
     (when-not project-id
       (ex/raise :type :validation
                 :code :missing-project
                 :hint "project not found"))
 
-    (let [path   (-> params :file :path)
-          format (bfc/parse-file-format path)
-          cfg    (assoc cfg
-                        ::bfc/profile-id profile-id
-                        ::bfc/project-id project-id
-                        ::bfc/input path)]
-
-      (if (= format :binfile-v3)
-        (bf.v3/import-files! cfg)
-        (bf.v1/import-files! cfg))
-
+    (let [path (-> params :file :path)
+          cfg  (assoc cfg
+                      ::bf.v1/overwrite overwrite?
+                      ::bf.v1/migrate migrate?
+                      ::bf.v1/profile-id profile-id
+                      ::bf.v1/project-id project-id
+                      ::bf.v1/input path)]
+      (bf.v1/import-files! cfg)
       {::yres/status  200
        ::yres/headers {"content-type" "text/plain"}
        ::yres/body    "OK"})))

backend/src/app/http/sse.clj

@@ -64,8 +64,7 @@
                           (catch Throwable cause
                             (events/tap :error (errors/handle' cause request))
                             (when-not (ex/instance? java.io.EOFException cause)
-                              (binding [l/*context* (errors/request->context request)]
-                                (l/err :hint "unexpected error on processing sse response" :cause cause))))
+                              (l/err :hint "unexpected error on processing sse response" :cause cause)))
                           (finally
                             (sp/close! events/*channel*)
                             (px/await! listener)))))))}))

backend/src/app/main.clj

@@ -349,6 +349,7 @@
      :file-gc            (ig/ref :app.tasks.file-gc/handler)
      :file-gc-scheduler  (ig/ref :app.tasks.file-gc-scheduler/handler)
      :offload-file-data  (ig/ref :app.tasks.offload-file-data/handler)
+     :file-xlog-gc       (ig/ref :app.tasks.file-xlog-gc/handler)
      :tasks-gc           (ig/ref :app.tasks.tasks-gc/handler)
      :telemetry          (ig/ref :app.tasks.telemetry/handler)
      :storage-gc-deleted (ig/ref ::sto.gc-deleted/handler)
@@ -404,6 +405,10 @@
    {::db/pool     (ig/ref ::db/pool)
     ::sto/storage (ig/ref ::sto/storage)}
 
+   :app.tasks.file-xlog-gc/handler
+   {::db/pool     (ig/ref ::db/pool)
+    ::sto/storage (ig/ref ::sto/storage)}
+
    :app.tasks.telemetry/handler
    {::db/pool            (ig/ref ::db/pool)
     ::http.client/client (ig/ref ::http.client/client)

backend/src/app/rpc/commands/auth.clj

@@ -273,8 +273,7 @@
                       (merge {:viewed-tutorial? false
                               :viewed-walkthrough? false
                               :nudge {:big 10 :small 1}
-                              :v2-info-shown true
-                              :release-notes-viewed (:main cf/version)})
+                              :v2-info-shown true})
                       (db/tjson))
 
         password  (or (:password params) "!")

backend/src/app/rpc/commands/binfile.clj

@@ -7,7 +7,6 @@
 (ns app.rpc.commands.binfile
   (:refer-clojure :exclude [assert])
   (:require
-   [app.binfile.common :as bfc]
    [app.binfile.v1 :as bf.v1]
    [app.binfile.v3 :as bf.v3]
    [app.common.logging :as l]
@@ -47,9 +46,9 @@
    (fn [_ output-stream]
      (try
        (-> cfg
-           (assoc ::bfc/ids #{file-id})
-           (assoc ::bfc/embed-assets embed-assets)
-           (assoc ::bfc/include-libraries include-libraries)
+           (assoc ::bf.v1/ids #{file-id})
+           (assoc ::bf.v1/embed-assets embed-assets)
+           (assoc ::bf.v1/include-libraries include-libraries)
            (bf.v1/export-files! output-stream))
        (catch Throwable cause
          (l/err :hint "exception on exporting file"
@@ -62,9 +61,9 @@
    (fn [_ output-stream]
      (try
        (-> cfg
-           (assoc ::bfc/ids #{file-id})
-           (assoc ::bfc/embed-assets embed-assets)
-           (assoc ::bfc/include-libraries include-libraries)
+           (assoc ::bf.v3/ids #{file-id})
+           (assoc ::bf.v3/embed-assets embed-assets)
+           (assoc ::bf.v3/include-libraries include-libraries)
            (bf.v3/export-files! output-stream))
        (catch Throwable cause
          (l/err :hint "exception on exporting file"
@@ -94,10 +93,10 @@
 (defn- import-binfile-v1
   [{:keys [::wrk/executor] :as cfg} {:keys [project-id profile-id name file]}]
   (let [cfg (-> cfg
-                (assoc ::bfc/project-id project-id)
-                (assoc ::bfc/profile-id profile-id)
-                (assoc ::bfc/name name)
-                (assoc ::bfc/input (:path file)))]
+                (assoc ::bf.v1/project-id project-id)
+                (assoc ::bf.v1/profile-id profile-id)
+                (assoc ::bf.v1/name name)
+                (assoc ::bf.v1/input (:path file)))]
 
     ;; NOTE: the importation process performs some operations that are
     ;; not very friendly with virtual threads, and for avoid
@@ -108,10 +107,10 @@
 (defn- import-binfile-v3
   [{:keys [::wrk/executor] :as cfg} {:keys [project-id profile-id name file]}]
   (let [cfg (-> cfg
-                (assoc ::bfc/project-id project-id)
-                (assoc ::bfc/profile-id profile-id)
-                (assoc ::bfc/name name)
-                (assoc ::bfc/input (:path file)))]
+                (assoc ::bf.v3/project-id project-id)
+                (assoc ::bf.v3/profile-id profile-id)
+                (assoc ::bf.v3/name name)
+                (assoc ::bf.v3/input (:path file)))]
     ;; NOTE: the importation process performs some operations that are
     ;; not very friendly with virtual threads, and for avoid
     ;; unexpected blocking of other concurrent operations we dispatch

backend/src/app/rpc/commands/comments.clj

@@ -29,11 +29,10 @@
 ;; --- GENERAL PURPOSE INTERNAL HELPERS
 
 (defn- decode-row
-  [{:keys [participants position mentions] :as row}]
+  [{:keys [participants position] :as row}]
   (cond-> row
     (db/pgpoint? position) (assoc :position (db/decode-pgpoint position))
-    (db/pgobject? participants) (assoc :participants (db/decode-transit-pgobject participants))
-    (db/pgarray? mentions) (assoc :mentions (db/decode-pgarray mentions #{}))))
+    (db/pgobject? participants) (assoc :participants (db/decode-transit-pgobject participants))))
 
 (def xf-decode-row
   (map decode-row))
@@ -462,9 +461,8 @@
                                              :thread-id thread-id
                                              :owner-id profile-id
                                              :content content})
-                        comment (decode-row comment)
-                        props   {:file-id file-id
-                                 :share-id nil}]
+                        props    {:file-id file-id
+                                  :share-id nil}]
 
                     ;; Update thread modified-at attribute and assoc the current
                     ;; profile to the participant set.

backend/src/app/rpc/commands/feedback.clj

@@ -17,7 +17,7 @@
    [app.rpc.doc :as-alias doc]
    [app.util.services :as sv]))
 
-(declare ^:private send-user-feedback!)
+(declare ^:private send-feedback!)
 
 (def ^:private schema:send-user-feedback
   [:map {:title "send-user-feedback"}
@@ -34,16 +34,14 @@
               :hint "feedback not enabled"))
 
   (let [profile (profile/get-profile pool profile-id)]
-    (send-user-feedback! pool profile params)
+    (send-feedback! pool profile params)
     nil))
 
-(defn- send-user-feedback!
+(defn- send-feedback!
   [pool profile params]
-  (let [dest (or (cf/get :user-feedback-destination)
-                 ;; LEGACY
-                 (cf/get :feedback-destination))]
+  (let [dest (cf/get :feedback-destination)]
     (eml/send! {::eml/conn pool
-                ::eml/factory eml/user-feedback
+                ::eml/factory eml/feedback
                 :from     dest
                 :to       dest
                 :profile  profile

backend/src/app/rpc/commands/files.clj

@@ -698,7 +698,11 @@
 
 (defn get-team-recent-files
   [conn team-id]
-  (db/exec! conn [sql:team-recent-files team-id]))
+  (->> (db/exec! conn [sql:team-recent-files team-id])
+       (mapv (fn [row]
+               (if-let [media-id (:thumbnail-id row)]
+                 (assoc row :thumbnail-uri (resolve-public-uri media-id))
+                 (dissoc row :media-id))))))
 
 (def ^:private schema:get-team-recent-files
   [:map {:title "get-team-recent-files"}

backend/src/app/rpc/commands/files_create.clj

@@ -118,12 +118,11 @@
         ;; feature on frontend and make it permanent on file
         features (-> (:features params #{})
                      (set/intersection cfeat/no-migration-features)
-                     (set/difference cfeat/frontend-only-features)
                      (set/union features))
 
         params   (-> params
                      (assoc :profile-id profile-id)
-                     (assoc :features (set/difference features cfeat/frontend-only-features)))]
+                     (assoc :features features))]
 
     (quotes/check! cfg {::quotes/id ::quotes/files-per-project
                         ::quotes/team-id team-id

backend/src/app/rpc/commands/files_snapshot.clj

@@ -28,19 +28,13 @@
    [cuerdas.core :as str]))
 
 (def sql:get-file-snapshots
-  "WITH changes AS (
-      SELECT id, label, revn, created_at, created_by, profile_id
-        FROM file_change
-       WHERE file_id = ?
-         AND data IS NOT NULL
-         AND (deleted_at IS NULL OR deleted_at > now())
-   ), versions AS (
-      (SELECT * FROM changes WHERE created_by = 'system' LIMIT 1000)
-      UNION ALL
-      (SELECT * FROM changes WHERE created_by != 'system' LIMIT 1000)
-   )
-   SELECT * FROM versions
-    ORDER BY created_at DESC;")
+  "SELECT id, label, revn, created_at, created_by, profile_id
+     FROM file_change
+    WHERE file_id = ?
+      AND data IS NOT NULL
+      AND (deleted_at IS NULL OR deleted_at > now())
+    ORDER BY created_at DESC
+    LIMIT 20")
 
 (defn get-file-snapshots
   [conn file-id]

backend/src/app/rpc/commands/files_thumbnails.clj

@@ -50,7 +50,8 @@
              " where file_id=? and tag=? and deleted_at is null")
         res (db/exec! conn [sql file-id tag])]
     (->> res
-         (d/index-by :object-id :media-id)
+         (d/index-by :object-id (fn [row]
+                                  (files/resolve-public-uri (:media-id row))))
          (d/without-nils))))
 
 (defn- get-object-thumbnails
@@ -61,7 +62,8 @@
               " where file_id=? and deleted_at is null")
          res (db/exec! conn [sql file-id])]
      (->> res
-          (d/index-by :object-id :media-id)
+          (d/index-by :object-id (fn [row]
+                                   (files/resolve-public-uri (:media-id row))))
           (d/without-nils))))
 
   ([conn file-id object-ids]
@@ -73,7 +75,8 @@
          res (db/exec! conn [sql file-id ids])]
 
      (->> res
-          (d/index-by :object-id :media-id)
+          (d/index-by :object-id (fn [row]
+                                   (files/resolve-public-uri (:media-id row))))
           (d/without-nils)))))
 
 (sv/defmethod ::get-file-object-thumbnails
@@ -124,11 +127,8 @@
               (if-let [frame  (-> frames first)]
                 (let [frame-id  (:id frame)
                       object-id (thc/fmt-object-id (:id file) page-id frame-id "frame")
-
-                      frame     (if-let [media-id (get thumbnails object-id)]
-                                  (-> frame
-                                      (assoc :thumbnail-id media-id)
-                                      (assoc :shapes []))
+                      frame     (if-let [thumb (get thumbnails object-id)]
+                                  (assoc frame :thumbnail thumb :shapes [])
                                   (dissoc frame :thumbnail))
 
                       children-ids
@@ -402,10 +402,7 @@
 
   [cfg {:keys [::rpc/profile-id file-id] :as params}]
   (db/tx-run! cfg (fn [{:keys [::db/conn] :as cfg}]
-                    ;; TODO For now we check read permissions instead of write,
-                    ;; to allow viewer users to update thumbnails. We might
-                    ;; review this approach on the future.
-                    (files/check-read-permissions! conn profile-id file-id)
+                    (files/check-edition-permissions! conn profile-id file-id)
                     (when-not (db/read-only? conn)
                       (let [media (create-file-thumbnail! cfg params)]
                         {:uri (files/resolve-public-uri (:id media))

backend/src/app/rpc/commands/files_update.clj

@@ -6,7 +6,6 @@
 
 (ns app.rpc.commands.files-update
   (:require
-   [app.binfile.common :as bfc]
    [app.common.data :as d]
    [app.common.exceptions :as ex]
    [app.common.features :as cfeat]
@@ -148,7 +147,7 @@
 
                           params   (-> params
                                        (assoc :profile-id profile-id)
-                                       (assoc :features (set/difference features cfeat/frontend-only-features))
+                                       (assoc :features features)
                                        (assoc :team team)
                                        (assoc :file file)
                                        (assoc :changes changes))
@@ -224,6 +223,15 @@
       (let [storage (sto/resolve cfg ::db/reuse-conn true)]
         (some->> (:data-ref-id file) (sto/touch-object! storage))))
 
+    (-> cfg
+        (assoc ::wrk/task :file-xlog-gc)
+        (assoc ::wrk/label (str "xlog:" (:id file)))
+        (assoc ::wrk/params {:file-id (:id file)})
+        (assoc ::wrk/delay (dt/duration "5m"))
+        (assoc ::wrk/dedupe true)
+        (assoc ::wrk/priority 1)
+        (wrk/submit!))
+
     (persist-file! cfg file)
 
     (let [params   (assoc params :file file)
@@ -416,37 +424,19 @@
       (l/error :hint "file validation error"
                :cause cause))))
 
-
 (defn- process-changes-and-validate
   [cfg file changes skip-validate]
   (let [;; WARNING: this ruins performance; maybe we need to find
         ;; some other way to do general validation
-        libs
-        (when (and (or (contains? cf/flags :file-validation)
-                       (contains? cf/flags :soft-file-validation))
-                   (not skip-validate))
-          (get-file-libraries cfg file))
+        libs (when (and (or (contains? cf/flags :file-validation)
+                            (contains? cf/flags :soft-file-validation))
+                        (not skip-validate))
+               (get-file-libraries cfg file))
 
-
-        ;; The main purpose of this atom is provide a contextual state
-        ;; for the changes subsystem where optionally some hints can
-        ;; be provided for the changes processing. Right now we are
-        ;; using it for notify about the existence of media refs when
-        ;; a new shape is added.
-        state
-        (atom {})
-
-        file
-        (binding [cpc/*state* state]
-          (-> (files/check-version! file)
-              (update :revn inc)
-              (update :data cpc/process-changes changes)
-              (update :data d/without-nils)))
-
-        file
-        (if-let [media-refs (-> @state :media-refs not-empty)]
-          (bfc/update-media-references! cfg file media-refs)
-          file)]
+        file (-> (files/check-version! file)
+                 (update :revn inc)
+                 (update :data cpc/process-changes changes)
+                 (update :data d/without-nils))]
 
     (binding [pmap/*tracked* nil]
       (when (contains? cf/flags :soft-file-validation)

backend/src/app/rpc/commands/management.clj

@@ -9,7 +9,6 @@
   (:require
    [app.binfile.common :as bfc]
    [app.binfile.v1 :as bf.v1]
-   [app.binfile.v3 :as bf.v3]
    [app.common.exceptions :as ex]
    [app.common.features :as cfeat]
    [app.common.schema :as sm]
@@ -26,7 +25,6 @@
    [app.rpc.doc :as-alias doc]
    [app.setup :as-alias setup]
    [app.setup.templates :as tmpl]
-   [app.storage.tmp :as tmp]
    [app.util.services :as sv]
    [app.util.time :as dt]
    [app.worker :as-alias wrk]
@@ -69,7 +67,7 @@
                      :is-owner true
                      :is-admin true
                      :can-edit true}
-                    {::db/return-keys false}))
+                    {::db/return-keys? false}))
 
       (doseq [params (sequence (comp
                                 (map #(bfc/remap-id % :file-id))
@@ -402,20 +400,11 @@
                     ;; that are not very friendly with virtual threads, and for
                     ;; avoid unexpected blocking of other concurrent operations
                     ;; we dispatch that operation to a dedicated executor.
-                    (let [template (tmp/tempfile-from template
-                                                      :prefix "penpot.template."
-                                                      :suffix ""
-                                                      :min-age "30m")
-                          format   (bfc/parse-file-format template)
-
-                          cfg      (-> cfg
-                                       (assoc ::bfc/project-id project-id)
-                                       (assoc ::bfc/profile-id profile-id)
-                                       (assoc ::bfc/input template))
-
-                          result   (if (= format :binfile-v3)
-                                     (px/invoke! executor (partial bf.v3/import-files! cfg))
-                                     (px/invoke! executor (partial bf.v1/import-files! cfg)))]
+                    (let [cfg    (-> cfg
+                                     (assoc ::bf.v1/project-id project-id)
+                                     (assoc ::bf.v1/profile-id profile-id)
+                                     (assoc ::bf.v1/input template))
+                          result (px/invoke! executor (partial bf.v1/import-files! cfg))]
 
                       (db/update! conn :project
                                   {:modified-at (dt/now)}

backend/src/app/rpc/commands/media.clj

@@ -60,25 +60,15 @@
   (media/validate-media-type! content)
   (media/validate-media-size! content)
 
-  (db/run! cfg (fn [{:keys [::db/conn] :as cfg}]
-                 ;; We get the minimal file for proper checking if
-                 ;; file is not already deleted
-                 (let [_     (files/get-minimal-file conn file-id)
-                       mobj  (create-file-media-object cfg params)]
-
-                   (db/update! conn :file
-                               {:modified-at (dt/now)
-                                :has-media-trimmed false}
-                               {:id file-id}
-                               {::db/return-keys false})
-
-                   (with-meta mobj
-                     {::audit/replace-props
-                      {:name (:name params)
-                       :file-id file-id
-                       :is-local (:is-local params)
-                       :size (:size content)
-                       :mtype (:mtype content)}})))))
+  (db/run! cfg (fn [cfg]
+                 (let [object (create-file-media-object cfg params)
+                       props  {:name (:name params)
+                               :file-id file-id
+                               :is-local (:is-local params)
+                               :size (:size content)
+                               :mtype (:mtype content)}]
+                   (with-meta object
+                     {::audit/replace-props props})))))
 
 (defn- big-enough-for-thumbnail?
   "Checks if the provided image info is big enough for
@@ -152,14 +142,20 @@
       :always
       (assoc ::image (process-main-image info)))))
 
-(defn- create-file-media-object
-  [{:keys [::sto/storage ::db/conn ::wrk/executor] :as cfg}
+(defn create-file-media-object
+  [{:keys [::sto/storage ::db/conn ::wrk/executor]}
    {:keys [id file-id is-local name content]}]
+
   (let [result (px/invoke! executor (partial process-image content))
         image  (sto/put-object! storage (::image result))
         thumb  (when-let [params (::thumb result)]
                  (sto/put-object! storage params))]
 
+    (db/update! conn :file
+                {:modified-at (dt/now)
+                 :has-media-trimmed false}
+                {:id file-id})
+
     (db/exec-one! conn [sql:create-file-media-object
                         (or id (uuid/next))
                         file-id is-local name
@@ -186,18 +182,7 @@
    ::sm/params schema:create-file-media-object-from-url}
   [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id] :as params}]
   (files/check-edition-permissions! pool profile-id file-id)
-  ;; We get the minimal file for proper checking if file is not
-  ;; already deleted
-  (let [_    (files/get-minimal-file cfg file-id)
-        mobj (create-file-media-object-from-url cfg (assoc params :profile-id profile-id))]
-
-    (db/update! pool :file
-                {:modified-at (dt/now)
-                 :has-media-trimmed false}
-                {:id file-id}
-                {::db/return-keys false})
-
-    mobj))
+  (create-file-media-object-from-url cfg (assoc params :profile-id profile-id)))
 
 (defn download-image
   [{:keys [::http/client]} uri]

backend/src/app/rpc/commands/profile.clj

@@ -422,9 +422,7 @@
                                   :deleted-at deleted-at
                                   :id profile-id}})
 
-
-      (-> (rph/wrap nil)
-          (rph/with-transform (session/delete-fn cfg))))))
+      (rph/with-transform {} (session/delete-fn cfg)))))
 
 
 ;; --- HELPERS
@@ -433,11 +431,8 @@
   "WITH owner_teams AS (
       SELECT tpr.team_id AS id
         FROM team_profile_rel AS tpr
-        JOIN team AS t ON (t.id = tpr.team_id)
        WHERE tpr.is_owner IS TRUE
          AND tpr.profile_id = ?
-         AND (t.deleted_at IS NULL OR
-              t.deleted_at > now())
    )
    SELECT tpr.team_id AS id,
           count(tpr.profile_id) - 1 AS participants

backend/src/app/rpc/commands/teams.clj

@@ -30,8 +30,7 @@
    [app.storage :as sto]
    [app.util.services :as sv]
    [app.util.time :as dt]
-   [app.worker :as wrk]
-   [clojure.set :as set]))
+   [app.worker :as wrk]))
 
 ;; --- Helpers & Specs
 
@@ -417,7 +416,6 @@
                       ::quotes/profile-id profile-id})
 
   (let [features (-> (cfeat/get-enabled-features cf/flags)
-                     (set/difference cfeat/frontend-only-features)
                      (cfeat/check-client-features! (:features params)))
         params   (-> params
                      (assoc :profile-id profile-id)

backend/src/app/rpc/commands/teams_invitations.clj

@@ -532,20 +532,17 @@
         team-owner (get-team-owner conn team-id)
 
         file       (when (some? file-id)
-                     (get-file-for-team-access-request cfg file-id))]
+                     (get-file-for-team-access-request cfg file-id))
 
-    (-> cfg
-        (assoc ::quotes/profile-id profile-id)
-        (assoc ::quotes/team-id team-id)
-        (quotes/check! {::quotes/id ::quotes/team-access-requests-per-team}
-                       {::quotes/id ::quotes/team-access-requests-per-requester}))
+        request    (upsert-team-access-request conn team-id profile-id)]
+
+    ;; FIXME missing quotes
 
     (teams/check-profile-muted conn requester)
     (teams/check-email-bounce conn (:email team-owner) false)
     (teams/check-email-spam conn (:email team-owner) true)
 
-    (let [request (upsert-team-access-request conn team-id profile-id)
-          factory (cond
+    (let [factory (cond
                     (and (some? file) (:is-default team) is-viewer)
                     eml/request-file-access-yourpenpot-view
 
@@ -568,9 +565,9 @@
                   :team-id team-id
                   :file-name (:name file)
                   :file-id file-id
-                  :page-id (:page-id file)})
+                  :page-id (:page-id file)}))
 
-      (with-meta {:request request}
-        {::audit/props {:request 1}}))))
+    (with-meta {:request request}
+      {::audit/props {:request 1}})))
 
 

backend/src/app/rpc/commands/verify_token.clj

@@ -166,26 +166,23 @@
         ;; invited team.
         (let [props {:team-id (:team-id claims)
                      :role (:role claims)
-                     :invitation-id (:id invitation)}]
+                     :invitation-id (:id invitation)}
 
-          (audit/submit!
-           cfg
-           (-> (audit/event-from-rpc-params params)
-               (assoc ::audit/name "accept-team-invitation")
-               (assoc ::audit/props props)))
+              accept-invitation-event
+              (-> (audit/event-from-rpc-params params)
+                  (assoc ::audit/name "accept-team-invitation")
+                  (assoc ::audit/props props))
 
-          ;; NOTE: Backward compatibility; old invitations can
-          ;; have the `created-by` to be nil; so in this case we
-          ;; don't submit this event to the audit-log
-          (when-let [created-by (:created-by invitation)]
-            (audit/submit!
-             cfg
-             (-> (audit/event-from-rpc-params params)
-                 (assoc ::audit/profile-id created-by)
-                 (assoc ::audit/name "accept-team-invitation-from")
-                 (assoc ::audit/props (assoc props
-                                             :profile-id (:id profile)
-                                             :email (:email profile))))))
+              accept-invitation-from-event
+              (-> (audit/event-from-rpc-params params)
+                  (assoc ::audit/profile-id (:created-by invitation))
+                  (assoc ::audit/name "accept-team-invitation-from")
+                  (assoc ::audit/props (assoc props
+                                              :profile-id (:id profile)
+                                              :email (:email profile))))]
+
+          (audit/submit! cfg accept-invitation-event)
+          (audit/submit! cfg accept-invitation-from-event)
 
           (accept-invitation cfg claims invitation profile)
           (assoc claims :state :created))

backend/src/app/rpc/doc.clj

@@ -87,7 +87,6 @@
       (let [params  (:query-params request)
             pstyle  (:type params "js")
             context (assoc context :param-style pstyle)]
-
         {::yres/status 200
          ::yres/body (-> (io/resource "app/templates/api-doc.tmpl")
                          (tmpl/render context))}))
@@ -208,7 +207,7 @@
   (assert (sm/valid? ::rpc/methods (::rpc/methods params)) "expected valid methods"))
 
 (defmethod ig/init-key ::routes
-  [_ {:keys [::rpc/methods] :as cfg}]
+  [_ {:keys [methods] :as cfg}]
   [(let [context (prepare-doc-context methods)]
      [["/_doc"
        {:handler (doc-handler context)

backend/src/app/rpc/quotes.clj

@@ -442,7 +442,7 @@
 
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;; QUOTE: SNAPSHOTS-PER-TEAM
+;; QUOTE: SNAPSHOTS-PER-FILE
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
 (def ^:private schema:snapshots-per-team
@@ -472,57 +472,6 @@
       (assoc ::count-sql [sql:get-snapshots-per-team team-id])
       (generic-check!)))
 
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;; QUOTE: TEAM-ACCESS-REQUESTS-PER-TEAM
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-
-(def ^:private schema:team-access-requests-per-team
-  [:map
-   [::profile-id ::sm/uuid]
-   [::team-id ::sm/uuid]])
-
-(def ^:private valid-team-access-requests-per-team-quote?
-  (sm/lazy-validator schema:team-access-requests-per-team))
-
-(def ^:private sql:get-team-access-requests-per-team
-  "SELECT count(*) AS total
-     FROM team_access_request AS tar
-    WHERE tar.team_id = ?")
-
-(defmethod check-quote ::team-access-requests-per-team
-  [{:keys [::profile-id ::team-id ::target] :as quote}]
-  (assert (valid-team-access-requests-per-team-quote? quote) "invalid quote parameters")
-  (-> quote
-      (assoc ::default (cf/get :quotes-team-access-requests-per-team Integer/MAX_VALUE))
-      (assoc ::quote-sql [sql:get-quotes-2 target team-id profile-id profile-id])
-      (assoc ::count-sql [sql:get-team-access-requests-per-team team-id])
-      (generic-check!)))
-
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;; QUOTE: TEAM-ACCESS-REQUESTS-PER-REQUESTER
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-
-(def ^:private schema:team-access-requests-per-requester
-  [:map
-   [::profile-id ::sm/uuid]])
-
-(def ^:private valid-team-access-requests-per-requester-quote?
-  (sm/lazy-validator schema:team-access-requests-per-requester))
-
-(def ^:private sql:get-team-access-requests-per-requester
-  "SELECT count(*) AS total
-     FROM team_access_request AS tar
-    WHERE tar.requester_id = ?")
-
-(defmethod check-quote ::team-access-requests-per-requester
-  [{:keys [::profile-id ::target] :as quote}]
-  (assert (valid-team-access-requests-per-requester-quote? quote) "invalid quote parameters")
-  (-> quote
-      (assoc ::default (cf/get :quotes-team-access-requests-per-requester Integer/MAX_VALUE))
-      (assoc ::quote-sql [sql:get-quotes-1 target profile-id])
-      (assoc ::count-sql [sql:get-team-access-requests-per-requester profile-id])
-      (generic-check!)))
-
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; QUOTE: DEFAULT
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

backend/src/app/setup.clj

@@ -74,7 +74,8 @@
 
 (defmethod ig/assert-key ::props
   [_ params]
-  (assert (db/pool? (::db/pool params)) "expected valid database pool"))
+  (assert (db/pool? (::db/pool params)) "expected valid database pool")
+  (assert (string? (::key params)) "expected valid key string"))
 
 (defmethod ig/init-key ::props
   [_ {:keys [::db/pool ::key] :as cfg}]

backend/src/app/setup/templates.clj

@@ -54,7 +54,6 @@
                               (::setup/templates cfg))]
     (let [dest (fs/join fs/*cwd* "builtin-templates")
           path (or (:path template) (fs/join dest template-id))]
-
       (if (fs/exists? path)
         (io/input-stream path)
         (let [resp (http/req! cfg

backend/src/app/srepl/fixes.clj

@@ -53,7 +53,7 @@
   fixes all not propertly referenced file-media-object for a file"
   [{:keys [id data] :as file} & _]
   (let [conn  (db/get-connection h/*system*)
-        used  (cfh/collect-used-media data)
+        used  (bfc/collect-used-media data)
         ids   (db/create-array conn "uuid" used)
         sql   (str "SELECT * FROM file_media_object WHERE id = ANY(?)")
         rows  (db/exec! conn [sql ids])

backend/src/app/storage/tmp.clj

@@ -16,13 +16,10 @@
    [app.util.time :as dt]
    [app.worker :as wrk]
    [datoteka.fs :as fs]
-   [datoteka.io :as io]
    [integrant.core :as ig]
    [promesa.exec :as px]
    [promesa.exec.csp :as sp])
   (:import
-   java.io.InputStream
-   java.io.OutputStream
    java.nio.file.Files))
 
 (def default-tmp-dir "/tmp/penpot")
@@ -89,12 +86,3 @@
     (fs/delete-on-exit! path)
     (sp/offer! queue [path (some-> min-age dt/duration)])
     path))
-
-(defn tempfile-from
-  "Create a new tempfile from from consuming the stream"
-  [input & {:as options}]
-  (let [path (tempfile options)]
-    (with-open [^InputStream input (io/input-stream input)]
-      (with-open [^OutputStream output (io/output-stream path)]
-        (io/copy input output)))
-    path))

backend/src/app/tasks/file_gc.clj

@@ -10,7 +10,7 @@
   file is eligible to be garbage collected after some period of
   inactivity (the default threshold is 72h)."
   (:require
-   [app.common.files.helpers :as cfh]
+   [app.binfile.common :as bfc]
    [app.common.files.migrations :as fmg]
    [app.common.files.validate :as cfv]
    [app.common.logging :as l]
@@ -26,6 +26,7 @@
    [app.util.pointer-map :as pmap]
    [app.util.time :as dt]
    [app.worker :as wrk]
+   [clojure.set :as set]
    [integrant.core :as ig]))
 
 (declare ^:private get-file)
@@ -52,21 +53,16 @@
    RETURNING id")
 
 (def ^:private xf:collect-used-media
-  (comp
-   (map :data)
-   (mapcat cfh/collect-used-media)))
+  (comp (map :data) (mapcat bfc/collect-used-media)))
 
 (defn- clean-file-media!
   "Performs the garbage collection of file media objects."
   [{:keys [::db/conn] :as cfg} {:keys [id] :as file}]
-  (let [xform  (comp
-                (map (partial decode-file cfg))
-                xf:collect-used-media)
-
-        used   (->> (db/plan conn [sql:get-snapshots id])
-                    (transduce xform conj #{}))
-        used   (into used xf:collect-used-media [file])
-
+  (let [used   (into #{}
+                     xf:collect-used-media
+                     (cons file
+                           (->> (db/cursor conn [sql:get-snapshots id])
+                                (map (partial decode-file cfg)))))
         ids    (db/create-array conn "uuid" used)
         unused (->> (db/exec! conn [sql:mark-file-media-object-deleted id ids])
                     (into #{} (map :id)))]
@@ -149,47 +145,51 @@
       AND f.deleted_at IS null
     ORDER BY f.modified_at ASC")
 
-(def ^:private xf:map-id (map :id))
-
-(defn- get-used-components
-  "Given a file and a set of components marked for deletion, return a
-  filtered set of component ids that are still un use"
-  [components library-id {:keys [data]}]
-  (filter #(ctf/used-in? data library-id % :component) components))
-
 (defn- clean-deleted-components!
   "Performs the garbage collection of unreferenced deleted components."
   [{:keys [::db/conn] :as cfg} {:keys [data] :as file}]
   (let [file-id (:id file)
 
-        deleted-components
-        (ctkl/deleted-components-seq data)
+        get-used-components
+        (fn [data components]
+          ;; Find which of the components are used in the file.
+          (into #{}
+                (filter #(ctf/used-in? data file-id % :component))
+                components))
 
-        xform
-        (mapcat (partial get-used-components deleted-components file-id))
+        get-unused-components
+        (fn [components files]
+          ;; Find and return a set of unused components (on all files).
+          (reduce (fn [components {:keys [data]}]
+                    (if (seq components)
+                      (->> (get-used-components data components)
+                           (set/difference components))
+                      (reduced components)))
 
-        used-remote
-        (->> (db/plan conn [sql:get-files-for-library file-id])
-             (transduce (comp (map (partial decode-file cfg)) xform) conj #{}))
+                  components
+                  files))
 
-        used-local
-        (into #{} xform [file])
+        process-fdata
+        (fn [data unused]
+          (reduce (fn [data id]
+                    (l/trc :hint "delete component"
+                           :component-id (str id)
+                           :file-id (str file-id))
+                    (ctkl/delete-component data id))
+                  data
+                  unused))
 
-        unused
-        (transduce xf:map-id disj
-                   (into #{} xf:map-id deleted-components)
-                   (concat used-remote used-local))
+        deleted (into #{} (ctkl/deleted-components-seq data))
+
+        unused  (->> (db/cursor conn [sql:get-files-for-library file-id] {:chunk-size 1})
+                     (map (partial decode-file cfg))
+                     (cons file)
+                     (get-unused-components deleted)
+                     (mapv :id)
+                     (set))
+
+        file    (update file :data process-fdata unused)]
 
-        file
-        (update file :data
-                (fn [data]
-                  (reduce (fn [data id]
-                            (l/trc :hint "delete component"
-                                   :component-id (str id)
-                                   :file-id (str file-id))
-                            (ctkl/delete-component data id))
-                          data
-                          unused)))]
 
     (l/dbg :hint "clean" :rel "components" :file-id (str file-id) :total (count unused))
     file))

backend/src/app/tasks/file_xlog_gc.clj

@@ -0,0 +1,64 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.tasks.file-xlog-gc
+  (:require
+   [app.common.logging :as l]
+   [app.config :as cf]
+   [app.db :as db]
+   [integrant.core :as ig]))
+
+;; Get the latest available snapshots without exceeding the total
+;; snapshot limit
+(def ^:private sql:get-latest-snapshots
+  "SELECT fch.id, fch.created_at
+     FROM file_change AS fch
+    WHERE fch.file_id = ?
+      AND fch.created_by = 'system'
+      AND fch.data IS NOT NULL
+      AND fch.deleted_at > now()
+    ORDER BY fch.created_at DESC
+    LIMIT ?")
+
+;; Mark all snapshots that are outside the allowed total threshold
+;; available for the GC
+(def ^:private sql:delete-snapshots
+  "UPDATE file_change
+      SET deleted_at = now()
+    WHERE file_id = ?
+      AND deleted_at > now()
+      AND data IS NOT NULL
+      AND created_by = 'system'
+      AND created_at < ?")
+
+(defn- get-alive-snapshots
+  [conn file-id]
+  (let [total     (cf/get :auto-file-snapshot-total 10)
+        snapshots (db/exec! conn [sql:get-latest-snapshots file-id total])]
+    (not-empty snapshots)))
+
+(defn- delete-old-snapshots!
+  [{:keys [::db/conn] :as cfg} file-id]
+  (when-let [snapshots (get-alive-snapshots conn file-id)]
+    (let [last-date (-> snapshots peek :created-at)
+          result    (db/exec-one! conn [sql:delete-snapshots file-id last-date])]
+      (l/inf :hint "delete old file snapshots"
+             :file-id (str file-id)
+             :current (count snapshots)
+             :deleted (db/get-update-count result)))))
+
+(defmethod ig/assert-key ::handler
+  [_ params]
+  (assert (db/pool? (::db/pool params)) "expected a valid database pool"))
+
+(defmethod ig/init-key ::handler
+  [_ cfg]
+  (fn [{:keys [props] :as task}]
+    (let [file-id (:file-id props)]
+      (assert (uuid? file-id) "expected file-id on props")
+      (-> cfg
+          (assoc ::db/rollback (:rollback props false))
+          (db/tx-run! delete-old-snapshots! file-id)))))

backend/test/backend_tests/binfile_test.clj

@@ -7,7 +7,6 @@
 (ns backend-tests.binfile-test
   "Internal binfile test, no RPC involved"
   (:require
-   [app.binfile.common :as bfc]
    [app.binfile.v3 :as v3]
    [app.common.features :as cfeat]
    [app.common.pprint :as pp]
@@ -94,15 +93,15 @@
 
     (v3/export-files!
      (-> th/*system*
-         (assoc ::bfc/ids #{(:id file)})
-         (assoc ::bfc/embed-assets false)
-         (assoc ::bfc/include-libraries false))
+         (assoc ::v3/ids #{(:id file)})
+         (assoc ::v3/embed-assets false)
+         (assoc ::v3/include-libraries false))
      (io/output-stream output))
 
     (let [result (-> th/*system*
-                     (assoc ::bfc/project-id (:default-project-id profile))
-                     (assoc ::bfc/profile-id (:id profile))
-                     (assoc ::bfc/input output)
+                     (assoc ::v3/project-id (:default-project-id profile))
+                     (assoc ::v3/profile-id (:id profile))
+                     (assoc ::v3/input output)
                      (v3/import-files!))]
       (t/is (= (count result) 1))
       (t/is (every? uuid? result)))))

backend/test/backend_tests/helpers.clj

@@ -62,7 +62,7 @@
 (def default
   {:database-uri "postgresql://postgres/penpot_test"
    :redis-uri "redis://redis/1"
-   :auto-file-snapshot-every 1})
+   :file-snapshot-every 1})
 
 (def config
   (cf/read-config :prefix "penpot-test"

backend/test/backend_tests/rpc_file_test.clj

@@ -383,19 +383,8 @@
       ;; as deleted.
       (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file)})))
 
-      ;; This only clears fragments, the file media objects still referenced because
-      ;; snapshots are preserved
       (let [res (th/run-task! :objects-gc {:min-age 0})]
-        (t/is (= 2 (:processed res))))
-
-      ;; Mark all snapshots to be a non-snapshot file change
-      (th/db-exec! ["update file_change set data = null where file_id = ?" (:id file)])
-      (th/db-exec! ["update file set has_media_trimmed = false where id = ?" (:id file)])
-
-      ;; Rerun the file-gc and objects-gc
-      (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file)})))
-      (let [res (th/run-task! :objects-gc {:min-age 0})]
-        (t/is (= 2 (:processed res))))
+        (t/is (= 3 (:processed res))))
 
       ;; Now that file-gc have deleted the file-media-object usage,
       ;; lets execute the touched-gc task, we should see that two of
@@ -428,6 +417,20 @@
 
               ;; (th/print-result! out)
               (t/is (nil? (:error out)))
+              (:result out)))
+
+          (update-file! [& {:keys [profile-id file-id changes revn] :or {revn 0}}]
+            (let [params {::th/type :update-file
+                          ::rpc/profile-id profile-id
+                          :id file-id
+                          :session-id (uuid/random)
+                          :revn revn
+                          :vern 0
+                          :components-v2 true
+                          :changes changes}
+                  out    (th/command! params)]
+              ;; (th/print-result! out)
+              (t/is (nil? (:error out)))
               (:result out)))]
 
     (let [storage (:app.storage/storage th/*system*)
@@ -547,20 +550,8 @@
       ;; as deleted.
       (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file)})))
 
-      ;; This only removes unused fragments, file media are still
-      ;; referenced on snapshots.
       (let [res (th/run-task! :objects-gc {:min-age 0})]
-        (t/is (= 2 (:processed res))))
-
-      ;; Mark all snapshots to be a non-snapshot file change
-      (th/db-exec! ["update file set has_media_trimmed = false where id = ?" (:id file)])
-      (th/db-exec! ["update file_change set data = null where file_id = ?" (:id file)])
-
-      ;; Rerun file-gc and objects-gc task for the same file once all snapshots are
-      ;; "expired/deleted"
-      (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file)})))
-      (let [res (th/run-task! :objects-gc {:min-age 0})]
-        (t/is (= 6 (:processed res))))
+        (t/is (= 7 (:processed res))))
 
       (let [rows (th/db-query :file-data-fragment {:file-id (:id file)
                                                    :deleted-at nil})]
@@ -600,7 +591,20 @@
 
               ;; (th/print-result! out)
               (t/is (nil? (:error out)))
-              (:result out)))]
+              (:result out)))
+
+          #_(update-file! [& {:keys [profile-id file-id changes revn] :or {revn 0}}]
+                          (let [params {::th/type :update-file
+                                        ::rpc/profile-id profile-id
+                                        :id file-id
+                                        :session-id (uuid/random)
+                                        :revn revn
+                                        :features cfeat/supported-features
+                                        :changes changes}
+                                out    (th/command! params)]
+              ;; (th/print-result! out)
+                            (t/is (nil? (:error out)))
+                            (:result out)))]
 
     (let [storage  (:app.storage/storage th/*system*)
           profile  (th/create-profile* 1)
@@ -1086,7 +1090,8 @@
         (t/is (contains? result :file-id))
 
         (t/is (= (:id file) (:file-id result)))
-        (t/is (uuid? (get-in result [:page :objects frame1-id :thumbnail-id])))
+        (t/is (str/starts-with? (get-in result [:page :objects frame1-id :thumbnail])
+                                "http://localhost:3449/assets/by-id/"))
         (t/is (= [] (get-in result [:page :objects frame1-id :shapes]))))
 
       ;; Delete thumbnail data
@@ -1332,500 +1337,3 @@
       (t/is (every? #(bytes? (:data %)) rows))
       (t/is (every? #(nil? (:data-ref-id %)) rows))
       (t/is (every? #(nil? (:data-backend %)) rows)))))
-
-(t/deftest file-gc-with-components-1
-  (let [storage (:app.storage/storage th/*system*)
-        profile (th/create-profile* 1)
-        file    (th/create-file* 1 {:profile-id (:id profile)
-                                    :project-id (:default-project-id profile)
-                                    :is-shared false})
-
-        s-id-1  (uuid/random)
-        s-id-2  (uuid/random)
-        c-id    (uuid/random)
-
-        page-id (first (get-in file [:data :pages]))]
-
-    (let [rows (th/db-query :file-data-fragment {:file-id (:id file)
-                                                 :deleted-at nil})]
-      (t/is (= (count rows) 1)))
-
-    ;; Update file inserting new component
-    (update-file!
-     :file-id (:id file)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :add-obj
-       :page-id page-id
-       :id s-id-1
-       :parent-id uuid/zero
-       :frame-id uuid/zero
-       :components-v2 true
-       :obj (cts/setup-shape
-             {:id s-id-1
-              :name "Board"
-              :frame-id uuid/zero
-              :parent-id uuid/zero
-              :type :frame
-              :main-instance true
-              :component-root true
-              :component-file (:id file)
-              :component-id c-id})}
-
-      {:type :add-obj
-       :page-id page-id
-       :id s-id-2
-       :parent-id uuid/zero
-       :frame-id uuid/zero
-       :components-v2 true
-       :obj (cts/setup-shape
-             {:id s-id-2
-              :name "Board"
-              :frame-id uuid/zero
-              :parent-id uuid/zero
-              :type :frame
-              :main-instance false
-              :component-root true
-              :component-file (:id file)
-              :component-id c-id})}
-
-      {:type :add-component
-       :path ""
-       :name "Board"
-       :main-instance-id s-id-1
-       :main-instance-page page-id
-       :id c-id
-       :anotation nil}])
-
-    ;; Run the file-gc task immediately without forced min-age
-    (t/is (false? (th/run-task! :file-gc {:file-id (:id file)})))
-
-    ;; Run the task again
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file)})))
-
-    ;; Retrieve file and check trimmed attribute
-    (let [row (th/db-get :file {:id (:id file)})]
-      (t/is (true? (:has-media-trimmed row))))
-
-    ;; Check that component exists
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result     (:result out)
-            component (get-in result [:data :components c-id])]
-
-        (t/is (some? component))
-        (t/is (nil? (:objects component)))))
-
-    ;; Now proceed to delete a component
-    (update-file!
-     :file-id (:id file)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :del-component
-       :id c-id}
-      {:type :del-obj
-       :page-id page-id
-       :id s-id-1
-       :ignore-touched true}])
-
-    ;; ;; Check that component is marked as deleted
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result (:result out)
-            component (get-in result [:data :components c-id])]
-        (t/is (true? (:deleted component)))
-        (t/is (some? (not-empty (:objects component))))))
-
-    ;; Re-run the file-gc task
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file)})))
-    (let [row (th/db-get :file {:id (:id file)})]
-      (t/is (true? (:has-media-trimmed row))))
-
-    ;; Check that component is still there after file-gc task
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result (:result out)
-            component (get-in result [:data :components c-id])]
-        (t/is (true? (:deleted component)))
-        (t/is (some? (not-empty (:objects component))))))
-
-    ;; Now delete the last instance using deleted component
-    (update-file!
-     :file-id (:id file)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :del-obj
-       :page-id page-id
-       :id s-id-2
-       :ignore-touched true}])
-
-    ;; Now, we have deleted the usage of component if we pass file-gc,
-    ;; that component should be deleted
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file)})))
-
-    ;; Check that component is properly removed
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result     (:result out)
-            components (get-in result [:data :components])]
-        (t/is (not (contains? components c-id)))))))
-
-(t/deftest file-gc-with-components-2
-  (let [storage (:app.storage/storage th/*system*)
-        profile (th/create-profile* 1)
-        file-1  (th/create-file* 1 {:profile-id (:id profile)
-                                    :project-id (:default-project-id profile)
-                                    :is-shared true})
-
-        file-2  (th/create-file* 2 {:profile-id (:id profile)
-                                    :project-id (:default-project-id profile)
-                                    :is-shared false})
-
-        rel     (th/link-file-to-library*
-                 {:file-id (:id file-2)
-                  :library-id (:id file-1)})
-
-        s-id-1  (uuid/random)
-        s-id-2  (uuid/random)
-        c-id    (uuid/random)
-
-        f1-page-id (first (get-in file-1 [:data :pages]))
-        f2-page-id (first (get-in file-2 [:data :pages]))]
-
-    ;; Update file library inserting new component
-    (update-file!
-     :file-id (:id file-1)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :add-obj
-       :page-id f1-page-id
-       :id s-id-1
-       :parent-id uuid/zero
-       :frame-id uuid/zero
-       :components-v2 true
-       :obj (cts/setup-shape
-             {:id s-id-1
-              :name "Board"
-              :frame-id uuid/zero
-              :parent-id uuid/zero
-              :type :frame
-              :main-instance true
-              :component-root true
-              :component-file (:id file-1)
-              :component-id c-id})}
-      {:type :add-component
-       :path ""
-       :name "Board"
-       :main-instance-id s-id-1
-       :main-instance-page f1-page-id
-       :id c-id
-       :anotation nil}])
-
-    ;; Instanciate a component in a different file
-    (update-file!
-     :file-id (:id file-2)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :add-obj
-       :page-id f2-page-id
-       :id s-id-2
-       :parent-id uuid/zero
-       :frame-id uuid/zero
-       :components-v2 true
-       :obj (cts/setup-shape
-             {:id s-id-2
-              :name "Board"
-              :frame-id uuid/zero
-              :parent-id uuid/zero
-              :type :frame
-              :main-instance false
-              :component-root true
-              :component-file (:id file-1)
-              :component-id c-id})}])
-
-    ;; Run the file-gc on file and library
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-1)})))
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-2)})))
-
-    ;; Check that component exists
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file-1)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result     (:result out)
-            component (get-in result [:data :components c-id])]
-
-        (t/is (some? component))
-        (t/is (nil? (:objects component)))))
-
-    ;; Now proceed to delete a component
-    (update-file!
-     :file-id (:id file-1)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :del-component
-       :id c-id}
-      {:type :del-obj
-       :page-id f1-page-id
-       :id s-id-1
-       :ignore-touched true}])
-
-    ;; Check that component is marked as deleted
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file-1)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result (:result out)
-            component (get-in result [:data :components c-id])]
-        (t/is (true? (:deleted component)))
-        (t/is (some? (not-empty (:objects component))))))
-
-    ;; Re-run the file-gc task
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-1)})))
-    (t/is (false? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-2)})))
-
-    ;; Check that component is still there after file-gc task
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file-1)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result (:result out)
-            component (get-in result [:data :components c-id])]
-        (t/is (true? (:deleted component)))
-        (t/is (some? (not-empty (:objects component))))))
-
-    ;; Now delete the last instance using deleted component
-    (update-file!
-     :file-id (:id file-2)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :del-obj
-       :page-id f2-page-id
-       :id s-id-2
-       :ignore-touched true}])
-
-    ;; Mark
-    (th/db-exec! ["update file set has_media_trimmed = false where id = ?" (:id file-1)])
-
-    ;; Now, we have deleted the usage of component if we pass file-gc,
-    ;; that component should be deleted
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-1)})))
-
-    ;; Check that component is properly removed
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file-1)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result     (:result out)
-            components (get-in result [:data :components])]
-        (t/is (not (contains? components c-id)))))))
-
-
-
-
-(defn add-file-media-object
-  [& {:keys [profile-id file-id]}]
-  (let [mfile  {:filename "sample.jpg"
-                :path (th/tempfile "backend_tests/test_files/sample.jpg")
-                :mtype "image/jpeg"
-                :size 312043}
-        params {::th/type :upload-file-media-object
-                ::rpc/profile-id profile-id
-                :file-id file-id
-                :is-local true
-                :name "testfile"
-                :content mfile}
-        out    (th/command! params)]
-
-    ;; (th/print-result! out)
-    (t/is (nil? (:error out)))
-    (:result out)))
-
-
-
-(t/deftest file-gc-with-media-assets-and-absorb-library
-  (let [storage (:app.storage/storage th/*system*)
-        profile (th/create-profile* 1)
-
-        file-1  (th/create-file* 1 {:profile-id (:id profile)
-                                    :project-id (:default-project-id profile)
-                                    :is-shared true})
-
-        file-2  (th/create-file* 2 {:profile-id (:id profile)
-                                    :project-id (:default-project-id profile)
-                                    :is-shared false})
-
-        fmedia  (add-file-media-object :profile-id (:id profile) :file-id (:id file-1))
-
-
-        rel     (th/link-file-to-library*
-                 {:file-id (:id file-2)
-                  :library-id (:id file-1)})
-
-        s-id-1  (uuid/random)
-        s-id-2  (uuid/random)
-        c-id    (uuid/random)
-
-        f1-page-id (first (get-in file-1 [:data :pages]))
-        f2-page-id (first (get-in file-2 [:data :pages]))
-
-        fills
-        [{:fill-image
-          {:id (:id fmedia)
-           :name "test"
-           :width 200
-           :height 200}}]]
-
-    ;; Update file library inserting new component
-    (update-file!
-     :file-id (:id file-1)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :add-obj
-       :page-id f1-page-id
-       :id s-id-1
-       :parent-id uuid/zero
-       :frame-id uuid/zero
-       :components-v2 true
-       :obj (cts/setup-shape
-             {:id s-id-1
-              :name "Board"
-              :frame-id uuid/zero
-              :parent-id uuid/zero
-              :type :frame
-              :fills fills
-              :main-instance true
-              :component-root true
-              :component-file (:id file-1)
-              :component-id c-id})}
-      {:type :add-component
-       :path ""
-       :name "Board"
-       :main-instance-id s-id-1
-       :main-instance-page f1-page-id
-       :id c-id
-       :anotation nil}])
-
-    ;; Instanciate a component in a different file
-    (update-file!
-     :file-id (:id file-2)
-     :profile-id (:id profile)
-     :revn 0
-     :vern 0
-     :changes
-     [{:type :add-obj
-       :page-id f2-page-id
-       :id s-id-2
-       :parent-id uuid/zero
-       :frame-id uuid/zero
-       :components-v2 true
-       :obj (cts/setup-shape
-             {:id s-id-2
-              :name "Board"
-              :frame-id uuid/zero
-              :parent-id uuid/zero
-              :type :frame
-              :fills fills
-              :main-instance false
-              :component-root true
-              :component-file (:id file-1)
-              :component-id c-id})}])
-
-    ;; Check that file media object references are present for both objects
-    ;; the original one and the instance.
-    (let [rows (th/db-exec! ["SELECT * FROM file_media_object ORDER BY created_at ASC"])]
-      (t/is (= 2 (count rows)))
-      (t/is (= (:id file-1) (:file-id (get rows 0))))
-      (t/is (= (:id file-2) (:file-id (get rows 1))))
-      (t/is (every? (comp nil? :deleted-at) rows)))
-
-    ;; Check if the underlying media reference on shape is different
-    ;; from the instantiation
-    (let [data {::th/type :get-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file-2)}
-          out  (th/command! data)]
-
-      (t/is (th/success? out))
-      (let [result (:result out)
-            fill   (get-in result [:data :pages-index f2-page-id :objects s-id-2 :fills 0 :fill-image])]
-        (t/is (some? fill))
-        (t/is (not= (:id fill) (:id fmedia)))))
-
-    ;; Run the file-gc on file and library
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-1)})))
-    (t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-2)})))
-
-    ;; Now proceed to delete file and absorb it
-    (let [data {::th/type :delete-file
-                ::rpc/profile-id (:id profile)
-                :id (:id file-1)}
-          out  (th/command! data)]
-      (t/is (th/success? out)))
-
-    (th/run-task! :delete-object
-                  {:object :file
-                   :deleted-at (dt/now)
-                   :id (:id file-1)})
-
-    ;; Check that file media object references are marked all for deletion
-    (let [rows (th/db-exec! ["SELECT * FROM file_media_object ORDER BY created_at ASC"])]
-      ;; (pp/pprint rows)
-      (t/is (= 2 (count rows)))
-
-      (t/is (= (:id file-1) (:file-id (get rows 0))))
-      (t/is (some? (:deleted-at (get rows 0))))
-
-      (t/is (= (:id file-2) (:file-id (get rows 1))))
-      (t/is (nil? (:deleted-at (get rows 1)))))
-
-    (th/run-task! :objects-gc
-                  {:min-age 0})
-
-    (let [rows (th/db-exec! ["SELECT * FROM file_media_object ORDER BY created_at ASC"])]
-      (t/is (= 1 (count rows)))
-
-      (t/is (= (:id file-2) (:file-id (get rows 0))))
-      (t/is (nil? (:deleted-at (get rows 0)))))))
-
-

backend/test/backend_tests/rpc_media_test.clj

@@ -10,7 +10,6 @@
    [app.db :as db]
    [app.rpc :as-alias rpc]
    [app.storage :as sto]
-   [app.util.time :as dt]
    [backend-tests.helpers :as th]
    [clojure.test :as t]
    [datoteka.fs :as fs]))
@@ -246,35 +245,3 @@
       (t/is (= "image/jpeg" (:mtype result)))
       (t/is (uuid? (:media-id result)))
       (t/is (uuid? (:thumbnail-id result))))))
-
-
-(t/deftest media-object-upload-command-when-file-is-deleted
-  (let [prof   (th/create-profile* 1)
-        proj   (th/create-project* 1 {:profile-id (:id prof)
-                                      :team-id (:default-team-id prof)})
-        file   (th/create-file* 1 {:profile-id (:id prof)
-                                   :project-id (:default-project-id prof)
-                                   :is-shared false})
-
-        _      (th/db-update! :file
-                              {:deleted-at (dt/now)}
-                              {:id (:id file)})
-
-        mfile  {:filename "sample.jpg"
-                :path (th/tempfile "backend_tests/test_files/sample.jpg")
-                :mtype "image/jpeg"
-                :size 312043}
-
-        params {::th/type :upload-file-media-object
-                ::rpc/profile-id (:id prof)
-                :file-id (:id file)
-                :is-local true
-                :name "testfile"
-                :content mfile}
-
-        out    (th/command! params)]
-
-    (let [error      (:error out)
-          error-data (ex-data error)]
-      (t/is (th/ex-info? error))
-      (t/is (= (:type error-data) :not-found)))))

backend/test/backend_tests/rpc_profile_test.clj

@@ -203,24 +203,7 @@
             edata (ex-data error)]
         (t/is (th/ex-info? error))
         (t/is (= (:type edata) :validation))
-        (t/is (= (:code edata) :owner-teams-with-people)))
-
-      (let [params {::th/type :delete-team
-                    ::rpc/profile-id (:id prof1)
-                    :id (:id team1)}
-            out    (th/command! params)]
-      ;; (th/print-result! out)
-
-        (let [team (th/db-get :team {:id (:id team1)} {::db/remove-deleted false})]
-          (t/is (dt/instant? (:deleted-at team)))))
-
-    ;; Request profile to be deleted
-      (let [params {::th/type :delete-profile
-                    ::rpc/profile-id (:id prof1)}
-            out    (th/command! params)]
-      ;; (th/print-result! out)
-        (t/is (nil? (:result out)))
-        (t/is (nil? (:error out)))))))
+        (t/is (= (:code edata) :owner-teams-with-people))))))
 
 (t/deftest profile-deletion-3
   (let [prof1 (th/create-profile* 1)
@@ -308,7 +291,7 @@
           out    (th/command! params)]
       ;; (th/print-result! out)
 
-      (t/is (nil? (:result out)))
+      (t/is (= {} (:result out)))
       (t/is (nil? (:error out))))
 
     ;; query files after profile soft deletion
@@ -353,7 +336,7 @@
                   ::rpc/profile-id (:id prof1)}
           out    (th/command! params)]
       ;; (th/print-result! out)
-      (t/is (nil? (:result out)))
+      (t/is (= {} (:result out)))
       (t/is (nil? (:error out))))
 
     (th/run-pending-tasks!)

common/package.json

@@ -1,11 +1,11 @@
 {
   "name": "common",
   "version": "1.0.0",
+  "main": "index.js",
   "license": "MPL-2.0",
   "author": "Kaleidos INC",
   "private": true,
   "packageManager": "yarn@4.3.1",
-  "type": "module",
   "repository": {
     "type": "git",
     "url": "https://github.com/penpot/penpot"
@@ -15,8 +15,6 @@
     "sax": "^1.4.1"
   },
   "devDependencies": {
-    "concurrently": "^9.0.1",
-    "nodemon": "^3.1.7",
     "shadow-cljs": "2.28.18",
     "source-map-support": "^0.5.21",
     "ws": "^8.17.0"
@@ -25,9 +23,9 @@
     "fmt:clj:check": "cljfmt check --parallel=false src/ test/",
     "fmt:clj": "cljfmt fix --parallel=true src/ test/",
     "lint:clj": "clj-kondo --parallel=true --lint src/",
-    "lint": "yarn run lint:clj",
-    "watch:test": "concurrently \"clojure -M:dev:shadow-cljs watch test\" \"nodemon -C -d 2 -w target/tests/ --exec 'node target/tests/test.js'\"",
-    "build:test": "clojure -M:dev:shadow-cljs compile test",
-    "test": "yarn run build:test && node target/tests/test.js"
+    "test:watch": "clojure -M:dev:shadow-cljs watch test",
+    "test:compile": "clojure -M:dev:shadow-cljs compile test --config-merge '{:autorun false}'",
+    "test:run": "node target/test.js",
+    "test": "yarn run test:compile && yarn run test:run"
   }
 }

common/shadow-cljs.edn

@@ -1,15 +1,19 @@
 {:deps {:aliases [:dev]}
  :builds
  {:test
-  {:target :esm
-   :output-dir "target/tests"
-   :runtime :node
-   :js-options {:js-provider :import}
+  {:target :node-test
+   :output-to "target/test.js"
+   :output-dir "target/test/"
+   :ns-regexp "^common-tests.*-test$"
+   :autorun true
 
-
-   :modules
-   {:test {:init-fn common-tests.runner/-main
-           :prepend-js "globalThis.navigator = {userAgent: \"\"}"}}}
+   :compiler-options
+   {:output-feature-set :es-next
+    :output-wrapper false
+    :source-map true
+    :source-map-include-sources-content true
+    :source-map-detail-level :all
+    :warnings {:fn-deprecated false}}}
 
   :bench
   {:target :node-script

common/src/app/common/features.cljc

@@ -59,8 +59,7 @@
   #{"fdata/shape-data-type"
     "styles/v2"
     "layout/grid"
-    "components/v2"
-    "plugins/runtime"})
+    "components/v2"})
 
 ;; A set of features which only affects on frontend and can be enabled
 ;; and disabled freely by the user any time. This features does not
@@ -155,7 +154,6 @@
         team-features    (into #{} xf-remove-ephimeral (:features team))]
     (-> enabled-features
         (set/intersection no-migration-features)
-        (set/difference frontend-only-features)
         (set/union team-features))))
 
 (defn check-client-features!

common/src/app/common/files/changes.cljc

@@ -484,11 +484,6 @@
   modification."
   nil)
 
-(def ^:dynamic *state*
-  "A general purpose state to signal some out of order operations
-  to the processor backend."
-  nil)
-
 (defmulti process-change (fn [_ change] (:type change)))
 (defmulti process-operation (fn [_ op] (:type op)))
 
@@ -622,38 +617,12 @@
 
 ;; --- Shape / Obj
 
-;; The main purpose of this is ensure that all created shapes has
-;; valid media references; so for make sure of it, we analyze each
-;; shape added via `:add-obj` change for media usage, and if shape has
-;; media refs, we put that media refs on the check list (on the
-;; *state*) which will subsequently be processed and all incorrect
-;; references will be corrected.  The media ref is anything that can
-;; be pointing to a file-media-object on the shape, per example we
-;; have fill-image, stroke-image, etc.
-
-(defn- collect-shape-media-refs
-  [state obj page-id]
-  (let [media-refs
-        (-> (cfh/collect-shape-media-refs obj)
-            (not-empty))
-
-        xform
-        (map (fn [id]
-               {:page-id page-id
-                :shape-id (:id obj)
-                :id id}))]
-
-    (update state :media-refs into xform media-refs)))
-
 (defmethod process-change :add-obj
   [data {:keys [id obj page-id component-id frame-id parent-id index ignore-touched]}]
   (let [update-container
         (fn [container]
           (ctst/add-shape id obj container frame-id parent-id index ignore-touched))]
 
-    (when *state*
-      (swap! *state* collect-shape-media-refs obj page-id))
-
     (if page-id
       (d/update-in-when data [:pages-index page-id] update-container)
       (d/update-in-when data [:components component-id] update-container))))
@@ -907,7 +876,7 @@
   (letfn [(update-fn [data]
             (if (some? value)
               (assoc-in data [:plugin-data namespace key] value)
-              (d/update-in-when data [:plugin-data namespace] dissoc key)))]
+              (update-in data [:plugin-data namespace] dissoc key)))]
 
     (case object-type
       :file

common/src/app/common/files/defaults.cljc

@@ -6,4 +6,4 @@
 
 (ns app.common.files.defaults)
 
-(def version 65)
+(def version 57)

common/src/app/common/files/helpers.cljc

@@ -12,7 +12,6 @@
    [app.common.schema :as sm]
    [app.common.uuid :as uuid]
    [clojure.set :as set]
-   [clojure.walk :as walk]
    [cuerdas.core :as str]))
 
 #?(:clj (set! *warn-on-reflection* true))
@@ -534,86 +533,6 @@
        (get-position-on-parent objects)
        inc))
 
-(defn collect-shape-media-refs
-  "Collect all media refs on the provided shape. Returns a set of ids"
-  [shape]
-  (sequence
-   (keep :id)
-   ;; NOTE: because of some bug, we ended with
-   ;; many shape types having the ability to
-   ;; have fill-image attribute (which initially
-   ;; designed for :path shapes).
-   (concat [(:fill-image shape)
-            (:metadata shape)]
-           (map :fill-image (:fills shape))
-           (map :stroke-image (:strokes shape))
-           (->> (:content shape)
-                (tree-seq map? :children)
-                (mapcat :fills)
-                (map :fill-image)))))
-
-(def ^:private
-  xform:collect-media-refs
-  "A transducer for collect media-id usage across a container (page or
-  component)"
-  (comp
-   (map :objects)
-   (mapcat vals)
-   (mapcat collect-shape-media-refs)))
-
-(defn collect-used-media
-  "Given a fdata (file data), returns all media references used in the
-  file data"
-  [data]
-  (-> #{}
-      (into xform:collect-media-refs (vals (:pages-index data)))
-      (into xform:collect-media-refs (vals (:components data)))
-      (into (keys (:media data)))))
-
-(defn relink-media-refs
-  "A function responsible to analyze all file data and replace the
-  old :component-file reference with the new ones, using the provided
-  file-index."
-  [data lookup-index]
-  (letfn [(process-map-form [form]
-            (cond-> form
-              ;; Relink image shapes
-              (and (map? (:metadata form))
-                   (= :image (:type form)))
-              (update-in [:metadata :id] lookup-index)
-
-              ;; Relink paths with fill image
-              (map? (:fill-image form))
-              (update-in [:fill-image :id] lookup-index)
-
-              ;; This covers old shapes and the new :fills.
-              (uuid? (:fill-color-ref-file form))
-              (update :fill-color-ref-file lookup-index)
-
-              ;; This covers the old shapes and the new :strokes
-              (uuid? (:stroke-color-ref-file form))
-              (update :stroke-color-ref-file lookup-index)
-
-              ;; This covers all text shapes that have typography referenced
-              (uuid? (:typography-ref-file form))
-              (update :typography-ref-file lookup-index)
-
-              ;; This covers the component instance links
-              (uuid? (:component-file form))
-              (update :component-file lookup-index)
-
-              ;; This covers the shadows and grids (they have directly
-              ;; the :file-id prop)
-              (uuid? (:file-id form))
-              (update :file-id lookup-index)))
-
-          (process-form [form]
-            (if (map? form)
-              (process-map-form form)
-              form))]
-
-    (walk/postwalk process-form data)))
-
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; SHAPES ORGANIZATION (PATH MANAGEMENT)
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

common/src/app/common/files/migrations.cljc

@@ -25,7 +25,6 @@
    [app.common.text :as txt]
    [app.common.types.color :as ctc]
    [app.common.types.component :as ctk]
-   [app.common.types.container :as ctn]
    [app.common.types.file :as ctf]
    [app.common.types.shape :as cts]
    [app.common.types.shape.shadow :as ctss]
@@ -1131,72 +1130,6 @@
         (update :pages-index dissoc nil)
         (update :pages-index update-vals update-page))))
 
-(defn migrate-up-59
-  [data]
-  (letfn [(fix-touched [elem]
-            (cond-> elem (string? elem) keyword))
-
-          (update-shape [shape]
-            (d/update-when shape :touched #(into #{} (map fix-touched) %)))
-
-          (update-container [container]
-            (d/update-when container :objects update-vals update-shape))]
-
-    (-> data
-        (update :pages-index update-vals update-container)
-        (update :components update-vals update-container))))
-
-(defn migrate-up-62
-  [data]
-  (let [xform-cycles-ids
-        (comp (filter #(= (:id %) (:shape-ref %)))
-              (map :id))
-
-        remove-cycles
-        (fn [objects]
-          (let [cycles-ids (into #{} xform-cycles-ids (vals objects))
-                to-detach  (->> cycles-ids
-                                (map #(get objects %))
-                                (map #(ctn/get-head-shape objects %))
-                                (map :id)
-                                distinct
-                                (mapcat #(ctn/get-children-in-instance objects %))
-                                (map :id)
-                                set)]
-
-            (reduce-kv (fn [objects id shape]
-                         (if (contains? to-detach id)
-                           (assoc objects id (ctk/detach-shape shape))
-                           objects))
-                       objects
-                       objects)))
-
-        update-component
-        (fn [component]
-          ;; we only have encounter this on deleted components,
-          ;; so the relevant objects are inside the component
-          (d/update-when component :objects remove-cycles))]
-
-    (update data :components update-vals update-component)))
-
-(defn migrate-up-65
-  [data]
-  (let [update-object
-        (fn [object]
-          (d/update-when object :plugin-data d/without-nils))
-
-        update-page
-        (fn [page]
-          (-> (update-object page)
-              (update :objects update-vals update-object)))]
-
-    (-> data
-        (update-object)
-        (d/update-when :pages-index update-vals update-page)
-        (d/update-when :colors update-vals update-object)
-        (d/update-when :typographies update-vals update-object)
-        (d/update-when :components update-vals update-object))))
-
 (def migrations
   "A vector of all applicable migrations"
   [{:id 2 :migrate-up migrate-up-2}
@@ -1245,7 +1178,5 @@
    {:id 54 :migrate-up migrate-up-54}
    {:id 55 :migrate-up migrate-up-55}
    {:id 56 :migrate-up migrate-up-56}
-   {:id 57 :migrate-up migrate-up-57}
-   {:id 59 :migrate-up migrate-up-59}
-   {:id 62 :migrate-up migrate-up-62}
-   {:id 65 :migrate-up migrate-up-65}])
+   {:id 57 :migrate-up migrate-up-57}])
+

common/src/app/common/files/repair.cljc

@@ -320,35 +320,6 @@
             (pcb/with-file-data file-data)
             (pcb/update-shapes shape-ids detach-shape))))))
 
-
-(defmethod repair-error :shape-ref-cycle
-  [_ {:keys [shape args] :as error} file-data _]
-  (let [repair-component
-        (fn [component]
-          (let [objects   (:objects component) ;; we only have encounter this on deleted components,
-                                               ;; so the relevant objects are inside the component
-                to-detach (->> (:cycles-ids args)
-                               (map #(get objects %))
-                               (map #(ctn/get-head-shape objects %))
-                               (map :id)
-                               distinct
-                               (mapcat #(ctn/get-children-in-instance objects %))
-                               (map :id)
-                               set)]
-
-            (update component :objects
-                    (fn [objects]
-                      (reduce-kv (fn [acc k v]
-                                   (if (contains? to-detach k)
-                                     (assoc acc k (ctk/detach-shape v))
-                                     (assoc acc k v)))
-                                 {}
-                                 objects)))))]
-    (log/dbg :hint "repairing component :shape-ref-cycle" :id (:id shape) :name (:name shape))
-    (-> (pcb/empty-changes nil nil)
-        (pcb/with-library-data file-data)
-        (pcb/update-component (:id shape) repair-component))))
-
 (defmethod repair-error :shape-ref-in-main
   [_ {:keys [shape page-id] :as error} file-data _]
   (let [repair-shape

common/src/app/common/files/validate.cljc

@@ -55,8 +55,7 @@
     :component-nil-objects-not-allowed
     :instance-head-not-frame
     :misplaced-slot
-    :missing-slot
-    :shape-ref-cycle})
+    :missing-slot})
 
 (def ^:private schema:error
   [:map {:title "ValidationError"}
@@ -483,18 +482,6 @@
                     "This deleted component has children with the same swap slot"
                     component file nil))))
 
-(defn check-ref-cycles
-  [component file]
-  (let [cycles-ids (->> component
-                        :objects
-                        vals
-                        (filter #(= (:id %) (:shape-ref %)))
-                        (map :id))]
-
-    (when (seq cycles-ids)
-      (report-error :shape-ref-cycle
-                    "This deleted component has shapes with shape-ref pointing to self"
-                    component file nil :cycles-ids cycles-ids))))
 
 (defn- check-component
   "Validate semantic coherence of a component. Report all errors found."
@@ -504,8 +491,7 @@
                   "Objects list cannot be nil"
                   component file nil))
   (when (:deleted component)
-    (check-component-duplicate-swap-slot component file)
-    (check-ref-cycles component file)))
+    (check-component-duplicate-swap-slot component file)))
 
 (defn- get-orphan-shapes
   [{:keys [objects] :as page}]

common/src/app/common/logic/libraries.cljc

@@ -304,12 +304,7 @@
                         (and (some? (ctk/get-swap-slot ref-shape))
                              (nil? (ctk/get-swap-slot shape))
                              (not= (:id shape) shape-id))
-                        (pcb/update-shapes [(:id shape)] #(ctk/set-swap-slot % (ctk/get-swap-slot ref-shape)))
-
-                        ;; If we can't get the ref-shape (e.g. it's in an external library not linked),
-                        ;: we can't do a suitable advance. So it's better to detach the shape
-                        (nil? ref-shape)
-                        (pcb/update-shapes [(:id shape)] ctk/detach-shape))))]
+                        (pcb/update-shapes [(:id shape)] #(ctk/set-swap-slot % (ctk/get-swap-slot ref-shape))))))]
 
     (reduce skip-near changes children)))
 

common/src/app/common/logic/shapes.cljc

@@ -304,9 +304,7 @@
                 (->> ids
                      (mapcat #(ctn/get-child-heads objects %))
                      (map :id)))
-
-        index-cell-data  (when to-index (ctl/get-cell-by-index parent to-index))
-        cell (or cell (and index-cell-data [(:row index-cell-data) (:column index-cell-data)]))]
+        cell (or cell (ctl/get-cell-by-index parent to-index))]
 
     (-> changes
         (pcb/with-page-id page-id)
@@ -411,14 +409,12 @@
         ;; Resize parent containers that need to
         (pcb/resize-parents parents))))
 
-(defn change-show-in-viewer
-  [shape hide?]
+(defn change-show-in-viewer [shape hide?]
   (assoc shape :hide-in-viewer hide?))
 
-(defn add-new-interaction
-  [shape interaction]
-  (update shape :interactions ctsi/add-interaction interaction))
+(defn add-new-interaction [shape interaction]
+  (-> shape
+      (update :interactions ctsi/add-interaction interaction)))
 
-(defn show-in-viewer
-  [shape]
+(defn show-in-viewer [shape]
   (dissoc shape :hide-in-viewer))

common/src/app/common/time.cljc

@@ -27,22 +27,10 @@
   #?(:clj (Instant/now)
      :cljs (.local ^js DateTime)))
 
-(defn is-after?
-  "Analgous to: da > db"
-  [da db]
-  (let [result (compare da db)]
-    (cond
-      (neg? result) false
-      (zero? result) false
-      :else true)))
-
-(defn is-before?
-  [da db]
-  (let [result (compare da db)]
-    (cond
-      (neg? result)   true
-      (zero? result)  false
-      :else false)))
+#?(:clj
+   (defn is-after?
+     [one other]
+     (.isAfter one other)))
 
 (defn instant?
   [o]

common/src/app/common/types/file.cljc

@@ -335,28 +335,24 @@
     (true? (= (:id component) (:id ref-component)))))
 
 (defn find-swap-slot
-  ([shape container file libraries]
-   (find-swap-slot shape container file libraries #{}))
-  ([shape container file libraries viewed-ids]
-   (if (contains? viewed-ids (:id shape)) ;; prevent cycles
-     nil
-     (if-let [swap-slot (ctk/get-swap-slot shape)]
-       swap-slot
-       (let [ref-shape (find-ref-shape file
-                                       container
-                                       libraries
-                                       shape
-                                       :include-deleted? true
-                                       :with-context? true)
-             shape-meta (meta ref-shape)
-             ref-file (:file shape-meta)
-             ref-container (:container shape-meta)]
-         (when ref-shape
-           (if-let [swap-slot (ctk/get-swap-slot ref-shape)]
-             swap-slot
-             (if (ctk/main-instance? ref-shape)
-               (:id shape)
-               (find-swap-slot ref-shape ref-container ref-file libraries (conj viewed-ids (:id shape)))))))))))
+  [shape container file libraries]
+  (if-let [swap-slot (ctk/get-swap-slot shape)]
+    swap-slot
+    (let [ref-shape (find-ref-shape file
+                                    container
+                                    libraries
+                                    shape
+                                    :include-deleted? true
+                                    :with-context? true)
+          shape-meta (meta ref-shape)
+          ref-file (:file shape-meta)
+          ref-container (:container shape-meta)]
+      (when ref-shape
+        (if-let [swap-slot (ctk/get-swap-slot ref-shape)]
+          swap-slot
+          (if (ctk/main-instance? ref-shape)
+            (:id shape)
+            (find-swap-slot ref-shape ref-container ref-file libraries)))))))
 
 (defn match-swap-slot?
   [shape-main shape-inst container-inst container-main file libraries]
@@ -742,20 +738,16 @@
                         (:component-id shape) "@"
                         :else "-")
 
-                  (when (:component-file shape)
+                  (when (and (:component-file shape) component-file)
                     (str/format "<%s> "
-                                (if component-file
-                                  (if (= (:id component-file) (:id file))
-                                    "local"
-                                    (:name component-file))
-                                  (if show-ids
-                                    (str/format "¿%s?" (:component-file shape))
-                                    "?"))))
+                                (if (= (:id component-file) (:id file))
+                                  "local"
+                                  (:name component-file))))
 
                   (or (:name component-shape)
-                      (if show-ids
-                        (str/format "¿%s?" (:shape-ref shape))
-                        "?"))
+                      (str/format "?%s"
+                                  (when show-ids
+                                    (str " " (:shape-ref shape)))))
 
                   (when (and show-ids component-shape)
                     (str/format " %s" (:id component-shape)))

common/src/app/common/types/modifiers.cljc

@@ -529,6 +529,13 @@
   (or (d/not-empty? (dm/get-prop modifiers :geometry-child))
       (d/not-empty? (dm/get-prop modifiers :structure-child))))
 
+(defn only-move?
+  "Returns true if there are only move operations"
+  [modifiers]
+  (let [move-op? #(= :move (dm/get-prop % :type))]
+    (and (every? move-op? (dm/get-prop modifiers :geometry-child))
+         (every? move-op? (dm/get-prop modifiers :geometry-parent)))))
+
 (defn has-geometry?
   [modifiers]
   (or (d/not-empty? (dm/get-prop modifiers :geometry-parent))
@@ -543,14 +550,6 @@
   [modifiers]
   (d/not-empty? (dm/get-prop modifiers :structure-child)))
 
-(defn only-move?
-  "Returns true if there are only move operations"
-  [modifiers]
-  (let [move-op? #(= :move (dm/get-prop % :type))]
-    (and (not (has-structure? modifiers))
-         (every? move-op? (dm/get-prop modifiers :geometry-child))
-         (every? move-op? (dm/get-prop modifiers :geometry-parent)))))
-
 ;; Extract subsets of modifiers
 
 (defn select-child

common/src/app/common/types/page.cljc

@@ -33,7 +33,7 @@
    [:id ::sm/uuid]
    [:axis [::sm/one-of #{:x :y}]]
    [:position ::sm/safe-number]
-   [:frame-id {:optional true} [:maybe ::sm/uuid]]])
+   [:frame-id {:optional true} ::sm/uuid]])
 
 (def schema:guides
   [:map-of {:gen/max 2} ::sm/uuid schema:guide])

common/src/app/common/types/shape.cljc

@@ -154,7 +154,6 @@
    [:main-instance {:optional true} :boolean]
    [:remote-synced {:optional true} :boolean]
    [:shape-ref {:optional true} ::sm/uuid]
-   [:touched {:optional true} [:maybe [:set :keyword]]]
    [:blocked {:optional true} :boolean]
    [:collapsed {:optional true} :boolean]
    [:locked {:optional true} :boolean]

common/src/app/common/types/shape/layout.cljc

@@ -1479,7 +1479,7 @@
 (defn get-cell-by-index
   [parent to-index]
   (let [cells (get-cells parent {:sort? true :remove-empty? true})
-        to-index (- (count cells) to-index 1)]
+        to-index (- (count cells) to-index)]
     (nth cells to-index nil)))
 
 (defn add-children-to-index

common/test/common_tests/logic/comp_detach_with_nested_test.cljc

@@ -103,83 +103,6 @@
     (t/is (= (:shape-ref copy-nested-ellipse) (thi/id :nested-ellipse)))
     (t/is (nil? (ctk/get-swap-slot copy-nested-ellipse)))))
 
-(t/deftest test-advance-in-library
-  (let [;; ==== Setup
-        library (setup-file)
-        file    (-> (thf/sample-file :file2)
-                    (thc/instantiate-component :c-big-board
-                                               :copy-big-board
-                                               :library library
-                                               :children-labels [:copy-h-board-with-ellipse
-                                                                 :copy-nested-h-ellipse
-                                                                 :copy-nested-ellipse]))
-        page (thf/current-page file)
-
-        ;; ==== Action
-        changes (cll/generate-detach-instance (-> (pcb/empty-changes nil)
-                                                  (pcb/with-page page)
-                                                  (pcb/with-objects (:objects page)))
-                                              page
-                                              {(:id file) file
-                                               (:id library) library}
-                                              (thi/id :copy-big-board))
-        file'   (thf/apply-changes file changes)
-
-        ;; ==== Get
-        copy-h-board-with-ellipse (ths/get-shape file' :copy-h-board-with-ellipse)
-        copy-nested-h-ellipse     (ths/get-shape file' :copy-nested-h-ellipse)
-        copy-nested-ellipse       (ths/get-shape file' :copy-nested-ellipse)]
-
-    ;; ==== Check
-
-    ;; It should the same as above, but in an external library.
-    (thf/dump-file file)
-    (t/is (ctk/instance-root? copy-h-board-with-ellipse))
-    (t/is (= (:shape-ref copy-h-board-with-ellipse) (thi/id :board-with-ellipse)))
-    (t/is (nil? (ctk/get-swap-slot copy-h-board-with-ellipse)))
-
-    (t/is (ctk/instance-head? copy-nested-h-ellipse))
-    (t/is (= (:shape-ref copy-nested-h-ellipse) (thi/id :nested-h-ellipse)))
-    (t/is (nil? (ctk/get-swap-slot copy-nested-h-ellipse)))
-
-    (t/is (not (ctk/instance-head? copy-nested-ellipse)))
-    (t/is (= (:shape-ref copy-nested-ellipse) (thi/id :nested-ellipse)))
-    (t/is (nil? (ctk/get-swap-slot copy-nested-ellipse)))))
-
-(t/deftest test-advance-in-broken-library
-  (let [;; ==== Setup
-        library (setup-file)
-        file    (-> (thf/sample-file :file2)
-                    (thc/instantiate-component :c-big-board
-                                               :copy-big-board
-                                               :library library
-                                               :children-labels [:copy-h-board-with-ellipse
-                                                                 :copy-nested-h-ellipse
-                                                                 :copy-nested-ellipse]))
-        page (thf/current-page file)
-
-        ;; ==== Action
-        changes (cll/generate-detach-instance (-> (pcb/empty-changes nil)
-                                                  (pcb/with-page page)
-                                                  (pcb/with-objects (:objects page)))
-                                              page
-                                              {(:id file) file}
-                                              (thi/id :copy-big-board))
-        file'   (thf/apply-changes file changes)
-
-        ;; ==== Get
-        copy-h-board-with-ellipse (ths/get-shape file' :copy-h-board-with-ellipse)
-        copy-nested-h-ellipse     (ths/get-shape file' :copy-nested-h-ellipse)
-        copy-nested-ellipse       (ths/get-shape file' :copy-nested-ellipse)]
-
-    ;; ==== Check
-
-    ;; If the main component cannot be found, because it's in a library that is
-    ;; not available, the nested copies should be detached too.
-    (t/is (not (ctk/in-component-copy? copy-h-board-with-ellipse)))
-    (t/is (not (ctk/in-component-copy? copy-nested-h-ellipse)))
-    (t/is (not (ctk/in-component-copy? copy-nested-ellipse)))))
-
 (t/deftest test-dont-advance-when-swapped-copy
   (let [;; ==== Setup
         file (-> (setup-file)

common/test/common_tests/runner.cljc

@@ -1,89 +0,0 @@
-;; This Source Code Form is subject to the terms of the Mozilla Public
-;; License, v. 2.0. If a copy of the MPL was not distributed with this
-;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
-;;
-;; Copyright (c) KALEIDOS INC
-
-(ns common-tests.runner
-  (:require
-   [clojure.test :as t]
-   [common-tests.colors-test]
-   [common-tests.data-test]
-   [common-tests.files-builder-test]
-   [common-tests.files-changes-test]
-   [common-tests.files-migrations-test]
-   [common-tests.geom-point-test]
-   [common-tests.geom-shapes-test]
-   [common-tests.geom-test]
-   [common-tests.logic.chained-propagation-test]
-   [common-tests.logic.comp-creation-test]
-   [common-tests.logic.comp-detach-with-nested-test]
-   [common-tests.logic.comp-remove-swap-slots-test]
-   [common-tests.logic.comp-reset-test]
-   [common-tests.logic.comp-sync-test]
-   [common-tests.logic.comp-touched-test]
-   [common-tests.logic.copying-and-duplicating-test]
-   [common-tests.logic.duplicated-pages-test]
-   [common-tests.logic.move-shapes-test]
-   [common-tests.logic.multiple-nesting-levels-test]
-   [common-tests.logic.swap-and-reset-test]
-   [common-tests.logic.swap-as-override-test]
-   [common-tests.pages-helpers-test]
-   [common-tests.record-test]
-   [common-tests.schema-test]
-   [common-tests.svg-path-test]
-   [common-tests.svg-test]
-   [common-tests.text-test]
-   [common-tests.time-test]
-   [common-tests.types-modifiers-test]
-   [common-tests.types-shape-interactions-test]
-   [common-tests.types.shape-decode-encode-test]
-   [common-tests.types.tokens-lib-test]
-   [common-tests.types.types-component-test]
-   [common-tests.types.types-libraries-test]))
-
-#?(:cljs (enable-console-print!))
-
-#?(:cljs
-   (defmethod cljs.test/report [:cljs.test/default :end-run-tests] [m]
-     (if (cljs.test/successful? m)
-       (.exit js/process 0)
-       (.exit js/process 1))))
-
-(defn -main
-  [& args]
-  (t/run-tests
-   'common-tests.colors-test
-   'common-tests.data-test
-   'common-tests.files-builder-test
-   'common-tests.files-changes-test
-   'common-tests.files-migrations-test
-   'common-tests.geom-point-test
-   'common-tests.geom-shapes-test
-   'common-tests.geom-test
-   'common-tests.logic.chained-propagation-test
-   'common-tests.logic.comp-creation-test
-   'common-tests.logic.comp-detach-with-nested-test
-   'common-tests.logic.comp-remove-swap-slots-test
-   'common-tests.logic.comp-reset-test
-   'common-tests.logic.comp-sync-test
-   'common-tests.logic.comp-touched-test
-   'common-tests.logic.copying-and-duplicating-test
-   'common-tests.logic.duplicated-pages-test
-   'common-tests.logic.move-shapes-test
-   'common-tests.logic.multiple-nesting-levels-test
-   'common-tests.logic.swap-and-reset-test
-   'common-tests.logic.swap-as-override-test
-   'common-tests.pages-helpers-test
-   'common-tests.record-test
-   'common-tests.schema-test
-   'common-tests.svg-path-test
-   'common-tests.svg-test
-   'common-tests.text-test
-   'common-tests.types-modifiers-test
-   'common-tests.types-shape-interactions-test
-   'common-tests.types.shape-decode-encode-test
-   'common-tests.types.types-component-test
-   'common-tests.types.types-libraries-test
-   'common-tests.types.tokens-lib-test
-   'common-tests.time-test))

common/test/common_tests/time_test.cljc

@@ -1,16 +0,0 @@
-;; This Source Code Form is subject to the terms of the Mozilla Public
-;; License, v. 2.0. If a copy of the MPL was not distributed with this
-;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
-;;
-;; Copyright (c) KALEIDOS INC
-
-(ns common-tests.time-test
-  (:require
-   [app.common.time :as dt]
-   [clojure.test :as t]))
-
-(t/deftest compare-time
-  (let [dta (dt/parse-instant 10000)
-        dtb (dt/parse-instant 20000)]
-    (t/is (false? (dt/is-after? dta dtb)))
-    (t/is (true? (dt/is-before? dta dtb)))))

common/test/common_tests/types/shape_decode_encode_test.cljc

@@ -148,4 +148,4 @@
          ;; (app.common.pprint/pprint shape)
          ;; (app.common.pprint/pprint shape-3)
          (= shape shape-3)))
-     {:num 100})))
+     {:num 1000})))

common/test/common_tests/types/tokens_lib_test.cljc

@@ -14,16 +14,8 @@
    [app.common.types.tokens-lib :as ctob]
    [clojure.test :as t]))
 
-(defn setup-virtual-time
-  [next]
-  (let [current (volatile! (inst-ms (dt/now)))]
-    (with-redefs [dt/now #(dt/parse-instant (vswap! current inc))]
-      (next))))
-
-(t/use-fixtures :once setup-virtual-time)
-
-(t/deftest tokens
-  (t/testing "make-token"
+(t/testing "token"
+  (t/deftest make-token
     (let [now    (dt/now)
           token1 (ctob/make-token :name "test-token-1"
                                   :type :boolean
@@ -48,14 +40,14 @@
       (t/is (= (:modified-at token2) now))
       (t/is (ctob/valid-token? token2))))
 
-  (t/testing "invalid-tokens"
+  (t/deftest invalid-tokens
     (let [args {:name 777
                 :type :invalid}]
-      (t/is (thrown-with-msg? #?(:cljs js/Error :clj Exception) #"expected valid token"
+      (t/is (thrown-with-msg? Exception #"expected valid token"
                               (apply ctob/make-token args)))
       (t/is (false? (ctob/valid-token? {})))))
 
-  (t/testing "find-token-value-references"
+  (t/deftest find-token-value-references
     (t/testing "finds references inside curly braces in a string"
       (t/is (= #{"foo" "bar"} (ctob/find-token-value-references "{foo} + {bar}")))
       (t/testing "ignores extra text"
@@ -65,8 +57,8 @@
     (t/testing "handles edge-case for extra curly braces"
       (t/is (= #{"foo" "bar"} (ctob/find-token-value-references "{foo}} + {bar}"))))))
 
-(t/deftest token-set
-  (t/testing "make-token-set"
+(t/testing "token-set"
+  (t/deftest make-token-set
     (let [now        (dt/now)
           token-set1 (ctob/make-token-set :name "test-token-set-1")
           token-set2 (ctob/make-token-set :name "test-token-set-2"
@@ -84,13 +76,13 @@
       (t/is (= (:modified-at token-set2) now))
       (t/is (empty? (:tokens token-set2)))))
 
-  (t/testing "invalid-token-set"
+  (t/deftest invalid-token-set
     (let [args {:name 777
                 :description 999}]
-      (t/is (thrown-with-msg? #?(:cljs js/Error :clj Exception) #"expected valid token set"
+      (t/is (thrown-with-msg? Exception #"expected valid token set"
                               (apply ctob/make-token-set args)))))
 
-  (t/testing "move-token-set"
+  (t/deftest move-token-set
     (let [tokens-lib (-> (ctob/make-tokens-lib)
                          (ctob/add-set (ctob/make-token-set :name "A"))
                          (ctob/add-set (ctob/make-token-set :name "B"))
@@ -115,7 +107,7 @@
         (t/is (= original-order (move "A" "foo/bar/baz")))
         (t/is (= original-order (move "Missing" "Move"))))))
 
-  (t/testing "tokens-tree"
+  (t/deftest tokens-tree
     (let [tokens-lib (-> (ctob/make-tokens-lib)
                          (ctob/add-set (ctob/make-token-set :name "A"
                                                             :tokens {"foo.bar.baz" (ctob/make-token :name "foo.bar.baz"
@@ -133,8 +125,8 @@
       (t/is (= (get-in expected ["foo" "bar" "bam" :name]) "foo.bar.bam"))
       (t/is (= (get-in expected ["baz" "boo" :name]) "baz.boo")))))
 
-(t/deftest token-theme
-  (t/testing "make-token-theme"
+(t/testing "token-theme"
+  (t/deftest make-token-theme
     (let [now          (dt/now)
           token-theme1 (ctob/make-token-theme :name "test-token-theme-1")
           token-theme2 (ctob/make-token-theme :name "test-token-theme-2"
@@ -158,24 +150,24 @@
       (t/is (= (:modified-at token-theme2) now))
       (t/is (empty? (:sets token-theme2)))))
 
-  (t/testing "invalid-token-theme"
+  (t/deftest invalid-token-theme
     (let [args {:name 777
                 :group nil
                 :description 999
                 :is-source 42}]
-      (t/is (thrown-with-msg? #?(:cljs js/Error :clj Exception) #"expected valid token theme"
+      (t/is (thrown-with-msg? Exception #"expected valid token theme"
                               (apply ctob/make-token-theme args))))))
 
 
-(t/deftest tokens-lib
-  (t/testing "make-tokens-lib"
+(t/testing "tokens-lib"
+  (t/deftest make-tokens-lib
     (let [tokens-lib (ctob/make-tokens-lib)]
       (t/is (= (ctob/set-count tokens-lib) 0))))
 
-  (t/testing "invalid-tokens-lib"
+  (t/deftest invalid-tokens-lib
     (let [args {:sets nil
                 :themes nil}]
-      (t/is (thrown-with-msg? #?(:cljs js/Error :clj Exception) #"expected valid tokens lib"
+      (t/is (thrown-with-msg? Exception #"expected valid tokens lib"
                               (apply ctob/make-tokens-lib args))))))
 
 
@@ -271,8 +263,8 @@
       (t/is (nil? token-set')))))
 
 
-(t/deftest token-in-a-lib
-  (t/testing "add-token"
+(t/testing "token in a lib"
+  (t/deftest add-token
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-set (ctob/make-token-set :name "test-token-set")))
           token       (ctob/make-token :name "test-token"
@@ -291,7 +283,7 @@
       (t/is (= (:name token') "test-token"))
       (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))))
 
-  (t/testing "update-token"
+  (t/deftest update-token
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                           (ctob/add-token-in-set "test-token-set"
@@ -332,7 +324,7 @@
       (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))
       (t/is (dt/is-after? (:modified-at token') (:modified-at token)))))
 
-  (t/testing "rename-token"
+  (t/deftest rename-token
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                           (ctob/add-token-in-set "test-token-set"
@@ -364,7 +356,7 @@
       (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))
       (t/is (dt/is-after? (:modified-at token') (:modified-at token)))))
 
-  (t/testing "delete-token"
+  (t/deftest delete-token
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                           (ctob/add-token-in-set "test-token-set"
@@ -385,7 +377,7 @@
       (t/is (nil? token'))
       (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))))
 
-  (t/testing "list-active-themes-tokens-in-order"
+  (t/deftest list-active-themes-tokens-in-order
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-theme (ctob/make-token-theme :name "out-of-order-theme"
                                                                  ;; Out of order sets in theme
@@ -413,8 +405,8 @@
       (t/is (= ["set-a-token" "set-b-token"] expected-token-names)))))
 
 
-(t/deftest token-theme-in-a-lib
-  (t/testing "add-token-theme"
+(t/testing "token-theme in a lib"
+  (t/deftest add-token-theme
     (let [tokens-lib  (ctob/make-tokens-lib)
           token-theme (ctob/make-token-theme :name "test-token-theme")
           tokens-lib' (ctob/add-theme tokens-lib token-theme)
@@ -426,7 +418,7 @@
       (t/is (= (first token-themes') token-theme))
       (t/is (= token-theme' token-theme))))
 
-  (t/testing "update-token-theme"
+  (t/deftest update-token-theme
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-theme (ctob/make-token-theme :name "test-token-theme")))
 
@@ -448,7 +440,7 @@
       (t/is (= (:description token-theme') "some description"))
       (t/is (dt/is-after? (:modified-at token-theme') (:modified-at token-theme)))))
 
-  (t/testing "rename-token-theme"
+  (t/deftest rename-token-theme
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-theme (ctob/make-token-theme :name "test-token-theme")))
 
@@ -465,7 +457,7 @@
       (t/is (= (:name token-theme') "updated-name"))
       (t/is (dt/is-after? (:modified-at token-theme') (:modified-at token-theme)))))
 
-  (t/testing "delete-token-theme"
+  (t/deftest delete-token-theme
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-theme (ctob/make-token-theme :name "test-token-theme")))
 
@@ -478,7 +470,7 @@
       (t/is (= (ctob/theme-count tokens-lib') 0))
       (t/is (nil? token-theme'))))
 
-  (t/testing "toggle-set-in-theme"
+  (t/deftest toggle-set-in-theme
     (let [tokens-lib   (-> (ctob/make-tokens-lib)
                            (ctob/add-set (ctob/make-token-set :name "token-set-1"))
                            (ctob/add-set (ctob/make-token-set :name "token-set-2"))
@@ -495,8 +487,8 @@
       (t/is (dt/is-after? (:modified-at token-theme') (:modified-at token-theme))))))
 
 
-(t/deftest serialization
-  (t/testing "transit-serialization"
+(t/testing "serialization"
+  (t/deftest transit-serialization
     (let [tokens-lib  (-> (ctob/make-tokens-lib)
                           (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                           (ctob/add-token-in-set "test-token-set" (ctob/make-token :name "test-token"
@@ -511,24 +503,23 @@
       (t/is (= (ctob/set-count tokens-lib') 1))
       (t/is (= (ctob/theme-count tokens-lib') 1))))
 
-  #?(:clj
-     (t/testing "fressian-serialization"
-       (let [tokens-lib   (-> (ctob/make-tokens-lib)
-                              (ctob/add-set (ctob/make-token-set :name "test-token-set"))
-                              (ctob/add-token-in-set "test-token-set" (ctob/make-token :name "test-token"
-                                                                                       :type :boolean
-                                                                                       :value true))
-                              (ctob/add-theme (ctob/make-token-theme :name "test-token-theme"))
-                              (ctob/toggle-set-in-theme "" "test-token-theme" "test-token-set"))
-             encoded-blob (fres/encode tokens-lib)
-             tokens-lib'  (fres/decode encoded-blob)]
+  (t/deftest fressian-serialization
+    (let [tokens-lib   (-> (ctob/make-tokens-lib)
+                           (ctob/add-set (ctob/make-token-set :name "test-token-set"))
+                           (ctob/add-token-in-set "test-token-set" (ctob/make-token :name "test-token"
+                                                                                    :type :boolean
+                                                                                    :value true))
+                           (ctob/add-theme (ctob/make-token-theme :name "test-token-theme"))
+                           (ctob/toggle-set-in-theme "" "test-token-theme" "test-token-set"))
+          encoded-blob (fres/encode tokens-lib)
+          tokens-lib'  (fres/decode encoded-blob)]
 
-         (t/is (ctob/valid-tokens-lib? tokens-lib'))
-         (t/is (= (ctob/set-count tokens-lib') 1))
-         (t/is (= (ctob/theme-count tokens-lib') 1))))))
+      (t/is (ctob/valid-tokens-lib? tokens-lib'))
+      (t/is (= (ctob/set-count tokens-lib') 1))
+      (t/is (= (ctob/theme-count tokens-lib') 1)))))
 
-(t/deftest grouping
-  (t/testing "split-and-join"
+(t/testing "grouping"
+  (t/deftest split-and-join
     (let [name "group/subgroup/name"
           path (ctob/split-path name "/")
           name' (ctob/join-path path "/")]
@@ -537,14 +528,14 @@
       (t/is (= (nth path 2) "name"))
       (t/is (= name' name))))
 
-  (t/testing "remove-spaces"
+  (t/deftest remove-spaces
     (let [name "group / subgroup / name"
           path (ctob/split-path name "/")]
       (t/is (= (first path) "group"))
       (t/is (= (second path) "subgroup"))
       (t/is (= (nth path 2) "name"))))
 
-  (t/testing "group-and-ungroup"
+  (t/deftest group-and-ungroup
     (let [token-set1   (ctob/make-token-set :name "token-set1")
           token-set2   (ctob/make-token-set :name "some group/token-set2")
 
@@ -557,7 +548,7 @@
       (t/is (= (:name token-set1'') "token-set1"))
       (t/is (= (:name token-set2'') "some group/token-set2"))))
 
-  (t/testing "get-groups-str"
+  (t/deftest get-groups-str
     (let [token-set1 (ctob/make-token-set :name "token-set1")
           token-set2 (ctob/make-token-set :name "some-group/token-set2")
           token-set3 (ctob/make-token-set :name "some-group/some-subgroup/token-set3")]
@@ -565,7 +556,7 @@
       (t/is (= (ctob/get-groups-str token-set2 "/") "some-group"))
       (t/is (= (ctob/get-groups-str token-set3 "/") "some-group/some-subgroup"))))
 
-  (t/testing "get-final-name"
+  (t/deftest get-final-name
     (let [token-set1 (ctob/make-token-set :name "token-set1")
           token-set2 (ctob/make-token-set :name "some-group/token-set2")
           token-set3 (ctob/make-token-set :name "some-group/some-subgroup/token-set3")]
@@ -574,7 +565,7 @@
       (t/is (= (ctob/get-final-name token-set3 "/") "token-set3"))))
 
   (t/testing "grouped tokens"
-    (t/testing "grouped-tokens"
+    (t/deftest grouped-tokens
       (let [tokens-lib (-> (ctob/make-tokens-lib)
                            (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                            (ctob/add-token-in-set "test-token-set"
@@ -608,7 +599,7 @@
         (t/is (= (:name (nth tokens-list 3)) "group1.subgroup11.token4"))
         (t/is (= (:name (nth tokens-list 4)) "group2.token5"))))
 
-    (t/testing "update-token-in-groups"
+    (t/deftest update-token-in-groups
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                             (ctob/add-token-in-set "test-token-set"
@@ -643,7 +634,7 @@
         (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))
         (t/is (dt/is-after? (:modified-at token') (:modified-at token)))))
 
-    (t/testing "rename-token-in-groups"
+    (t/deftest rename-token-in-groups
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                             (ctob/add-token-in-set "test-token-set"
@@ -677,7 +668,7 @@
         (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))
         (t/is (dt/is-after? (:modified-at token') (:modified-at token)))))
 
-    (t/testing "move-token-of-group"
+    (t/deftest move-token-of-group
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                             (ctob/add-token-in-set "test-token-set"
@@ -712,7 +703,7 @@
         (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))
         (t/is (dt/is-after? (:modified-at token') (:modified-at token)))))
 
-    (t/testing "delete-token-in-group"
+    (t/deftest delete-token-in-group
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "test-token-set"))
                             (ctob/add-token-in-set "test-token-set"
@@ -736,7 +727,7 @@
         (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set))))))
 
   (t/testing "grouped sets"
-    (t/testing "grouped-sets"
+    (t/deftest grouped-sets
       (let [tokens-lib (-> (ctob/make-tokens-lib)
                            (ctob/add-set (ctob/make-token-set :name "token-set-1"))
                            (ctob/add-set (ctob/make-token-set :name "group1/token-set-2"))
@@ -795,7 +786,7 @@
         (t/is (= (ctob/group? (second node-set5)) false))
         (t/is (= (:name (second node-set5)) "group2/token-set-5"))))
 
-    (t/testing "update-set-in-groups"
+    (t/deftest update-set-in-groups
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "token-set-1"))
                             (ctob/add-set (ctob/make-token-set :name "group1/token-set-2"))
@@ -821,7 +812,7 @@
         (t/is (= (:description token-set') "some description"))
         (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))))
 
-    (t/testing "rename-set-in-groups"
+    (t/deftest rename-set-in-groups
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "token-set-1"))
                             (ctob/add-set (ctob/make-token-set :name "group1/token-set-2"))
@@ -848,7 +839,7 @@
         (t/is (= (:description token-set') nil))
         (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))))
 
-    (t/testing "move-set-of-group"
+    (t/deftest move-set-of-group
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "token-set-1"))
                             (ctob/add-set (ctob/make-token-set :name "group1/token-set-2"))
@@ -877,7 +868,7 @@
         (t/is (= (:description token-set') nil))
         (t/is (dt/is-after? (:modified-at token-set') (:modified-at token-set)))))
 
-    (t/testing "delete-set-in-group"
+    (t/deftest delete-set-in-group
       (let [tokens-lib  (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "token-set-1"))
                             (ctob/add-set (ctob/make-token-set :name "group1/token-set-2")))
@@ -893,7 +884,7 @@
         (t/is (nil? token-set')))))
 
   (t/testing "grouped themes"
-    (t/testing "grouped-themes"
+    (t/deftest grouped-themes
       (let [tokens-lib (-> (ctob/make-tokens-lib)
                            (ctob/add-theme (ctob/make-token-theme :group "" :name "token-theme-1"))
                            (ctob/add-theme (ctob/make-token-theme :group "group1" :name "token-theme-2"))
@@ -950,7 +941,7 @@
         (t/is (= (ctob/group? (second node-theme4)) false))
         (t/is (= (:name (second node-theme4)) "token-theme-4"))))
 
-    (t/testing "update-theme-in-groups"
+    (t/deftest update-theme-in-groups
       (let [tokens-lib   (-> (ctob/make-tokens-lib)
                              (ctob/add-theme (ctob/make-token-theme :group "" :name "token-theme-1"))
                              (ctob/add-theme (ctob/make-token-theme :group "group1" :name "token-theme-2"))
@@ -976,7 +967,7 @@
         (t/is (= (:description token-theme') "some description"))
         (t/is (dt/is-after? (:modified-at token-theme') (:modified-at token-theme)))))
 
-    (t/testing "get-theme-groups"
+    (t/deftest get-theme-groups
       (let [token-lib (-> (ctob/make-tokens-lib)
                           (ctob/add-theme (ctob/make-token-theme :group "" :name "token-theme-1"))
                           (ctob/add-theme (ctob/make-token-theme :group "group1" :name "token-theme-2"))
@@ -985,14 +976,13 @@
             token-groups (ctob/get-theme-groups token-lib)]
         (t/is (= token-groups ["group1" "group2"]))))
 
-    (t/testing "rename-theme-in-groups"
+    (t/deftest rename-theme-in-groups
       (let [tokens-lib   (-> (ctob/make-tokens-lib)
                              (ctob/add-theme (ctob/make-token-theme :group "" :name "token-theme-1"))
                              (ctob/add-theme (ctob/make-token-theme :group "group1" :name "token-theme-2"))
                              (ctob/add-theme (ctob/make-token-theme :group "group1" :name "token-theme-3"))
                              (ctob/add-theme (ctob/make-token-theme :group "group2" :name "token-theme-4")))
 
-
             tokens-lib'  (-> tokens-lib
                              (ctob/update-theme "group1" "token-theme-2"
                                                 (fn [token-theme]
@@ -1013,7 +1003,7 @@
         (t/is (= (:description token-theme') nil))
         (t/is (dt/is-after? (:modified-at token-theme') (:modified-at token-theme)))))
 
-    (t/testing "move-theme-of-group"
+    (t/deftest move-theme-of-group
       (let [tokens-lib   (-> (ctob/make-tokens-lib)
                              (ctob/add-theme (ctob/make-token-theme :group "" :name "token-theme-1"))
                              (ctob/add-theme (ctob/make-token-theme :group "group1" :name "token-theme-2"))
@@ -1043,7 +1033,7 @@
         (t/is (= (:description token-theme') nil))
         (t/is (dt/is-after? (:modified-at token-theme') (:modified-at token-theme)))))
 
-    (t/testing "delete-theme-in-group"
+    (t/deftest delete-theme-in-group
       (let [tokens-lib   (-> (ctob/make-tokens-lib)
                              (ctob/add-theme (ctob/make-token-theme :group "" :name "token-theme-1"))
                              (ctob/add-theme (ctob/make-token-theme :group "group1" :name "token-theme-2")))
@@ -1059,8 +1049,8 @@
         (t/is (nil? token-theme'))))))
 
 #?(:clj
-   (t/deftest dtcg-encoding-decoding
-     (t/testing "decode-dtcg-json"
+   (t/testing "dtcg encoding/decoding"
+     (t/deftest decode-dtcg-json
        (let [json (-> (slurp "test/common_tests/types/data/tokens-multi-set-example.json")
                       (tr/decode-str))
              lib (ctob/decode-dtcg-json (ctob/ensure-tokens-lib nil) json)
@@ -1088,7 +1078,7 @@
          (t/testing "invalid tokens got discarded"
            (t/is (nil? (get-set-token "typography" "H1.Bold"))))))
 
-     (t/testing "encode-dtcg-json"
+     (t/deftest encode-dtcg-json
        (let [tokens-lib (-> (ctob/make-tokens-lib)
                             (ctob/add-set (ctob/make-token-set :name "core"
                                                                :tokens {"colors.red.600"
@@ -1121,7 +1111,7 @@
                                               "$type" "color"}}}}}
                   expected))))
 
-     (t/testing "encode-decode-dtcg-json"
+     (t/deftest encode-decode-dtcg-json
        (with-redefs [dt/now (constantly #inst "2024-10-16T12:01:20.257840055-00:00")]
          (let [tokens-lib (-> (ctob/make-tokens-lib)
                               (ctob/add-set (ctob/make-token-set :name "core"

common/yarn.lock

@@ -88,7 +88,7 @@ __metadata:
   languageName: node
   linkType: hard
 
-"ansi-styles@npm:^4.0.0, ansi-styles@npm:^4.1.0":
+"ansi-styles@npm:^4.0.0":
   version: 4.3.0
   resolution: "ansi-styles@npm:4.3.0"
   dependencies:
@@ -104,16 +104,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"anymatch@npm:~3.1.2":
-  version: 3.1.3
-  resolution: "anymatch@npm:3.1.3"
-  dependencies:
-    normalize-path: "npm:^3.0.0"
-    picomatch: "npm:^2.0.4"
-  checksum: 10c0/57b06ae984bc32a0d22592c87384cd88fe4511b1dd7581497831c56d41939c8a001b28e7b853e1450f2bf61992dfcaa8ae2d0d161a0a90c4fb631ef07098fbac
-  languageName: node
-  linkType: hard
-
 "asn1.js@npm:^4.10.1":
   version: 4.10.1
   resolution: "asn1.js@npm:4.10.1"
@@ -149,13 +139,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"binary-extensions@npm:^2.0.0":
-  version: 2.3.0
-  resolution: "binary-extensions@npm:2.3.0"
-  checksum: 10c0/75a59cafc10fb12a11d510e77110c6c7ae3f4ca22463d52487709ca7f18f69d886aa387557cc9864fbdb10153d0bdb4caacabf11541f55e89ed6e18d12ece2b5
-  languageName: node
-  linkType: hard
-
 "bn.js@npm:^4.0.0, bn.js@npm:^4.1.0, bn.js@npm:^4.11.9":
   version: 4.12.0
   resolution: "bn.js@npm:4.12.0"
@@ -170,16 +153,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"brace-expansion@npm:^1.1.7":
-  version: 1.1.11
-  resolution: "brace-expansion@npm:1.1.11"
-  dependencies:
-    balanced-match: "npm:^1.0.0"
-    concat-map: "npm:0.0.1"
-  checksum: 10c0/695a56cd058096a7cb71fb09d9d6a7070113c7be516699ed361317aca2ec169f618e28b8af352e02ab4233fb54eb0168460a40dc320bab0034b36ab59aaad668
-  languageName: node
-  linkType: hard
-
 "brace-expansion@npm:^2.0.1":
   version: 2.0.1
   resolution: "brace-expansion@npm:2.0.1"
@@ -189,15 +162,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"braces@npm:~3.0.2":
-  version: 3.0.3
-  resolution: "braces@npm:3.0.3"
-  dependencies:
-    fill-range: "npm:^7.1.1"
-  checksum: 10c0/7c6dfd30c338d2997ba77500539227b9d1f85e388a5f43220865201e407e076783d0881f2d297b9f80951b4c957fcf0b51c1d2d24227631643c3f7c284b0aa04
-  languageName: node
-  linkType: hard
-
 "brorand@npm:^1.0.1, brorand@npm:^1.1.0":
   version: 1.1.0
   resolution: "brorand@npm:1.1.0"
@@ -344,35 +308,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"chalk@npm:^4.1.2":
-  version: 4.1.2
-  resolution: "chalk@npm:4.1.2"
-  dependencies:
-    ansi-styles: "npm:^4.1.0"
-    supports-color: "npm:^7.1.0"
-  checksum: 10c0/4a3fef5cc34975c898ffe77141450f679721df9dde00f6c304353fa9c8b571929123b26a0e4617bde5018977eb655b31970c297b91b63ee83bb82aeb04666880
-  languageName: node
-  linkType: hard
-
-"chokidar@npm:^3.5.2":
-  version: 3.6.0
-  resolution: "chokidar@npm:3.6.0"
-  dependencies:
-    anymatch: "npm:~3.1.2"
-    braces: "npm:~3.0.2"
-    fsevents: "npm:~2.3.2"
-    glob-parent: "npm:~5.1.2"
-    is-binary-path: "npm:~2.1.0"
-    is-glob: "npm:~4.0.1"
-    normalize-path: "npm:~3.0.0"
-    readdirp: "npm:~3.6.0"
-  dependenciesMeta:
-    fsevents:
-      optional: true
-  checksum: 10c0/8361dcd013f2ddbe260eacb1f3cb2f2c6f2b0ad118708a343a5ed8158941a39cb8fb1d272e0f389712e74ee90ce8ba864eece9e0e62b9705cb468a2f6d917462
-  languageName: node
-  linkType: hard
-
 "chownr@npm:^2.0.0":
   version: 2.0.0
   resolution: "chownr@npm:2.0.0"
@@ -397,17 +332,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"cliui@npm:^8.0.1":
-  version: 8.0.1
-  resolution: "cliui@npm:8.0.1"
-  dependencies:
-    string-width: "npm:^4.2.0"
-    strip-ansi: "npm:^6.0.1"
-    wrap-ansi: "npm:^7.0.0"
-  checksum: 10c0/4bda0f09c340cbb6dfdc1ed508b3ca080f12992c18d68c6be4d9cf51756033d5266e61ec57529e610dacbf4da1c634423b0c1b11037709cc6b09045cbd815df5
-  languageName: node
-  linkType: hard
-
 "color-convert@npm:^2.0.1":
   version: 2.0.1
   resolution: "color-convert@npm:2.0.1"
@@ -428,41 +352,14 @@ __metadata:
   version: 0.0.0-use.local
   resolution: "common@workspace:."
   dependencies:
-    concurrently: "npm:^9.0.1"
     luxon: "npm:^3.4.4"
-    nodemon: "npm:^3.1.7"
     sax: "npm:^1.4.1"
-    shadow-cljs: "npm:2.28.18"
+    shadow-cljs: "npm:2.28.11"
     source-map-support: "npm:^0.5.21"
     ws: "npm:^8.17.0"
   languageName: unknown
   linkType: soft
 
-"concat-map@npm:0.0.1":
-  version: 0.0.1
-  resolution: "concat-map@npm:0.0.1"
-  checksum: 10c0/c996b1cfdf95b6c90fee4dae37e332c8b6eb7d106430c17d538034c0ad9a1630cb194d2ab37293b1bdd4d779494beee7786d586a50bd9376fd6f7bcc2bd4c98f
-  languageName: node
-  linkType: hard
-
-"concurrently@npm:^9.0.1":
-  version: 9.1.0
-  resolution: "concurrently@npm:9.1.0"
-  dependencies:
-    chalk: "npm:^4.1.2"
-    lodash: "npm:^4.17.21"
-    rxjs: "npm:^7.8.1"
-    shell-quote: "npm:^1.8.1"
-    supports-color: "npm:^8.1.1"
-    tree-kill: "npm:^1.2.2"
-    yargs: "npm:^17.7.2"
-  bin:
-    conc: dist/bin/concurrently.js
-    concurrently: dist/bin/concurrently.js
-  checksum: 10c0/f2f42f94dde508bfbaf47b5ac654db9e8a4bf07d3d7b6267dd058ae6f362eec677ae7c8ede398d081e5fd0d1de5811dc9a53e57d3f1f68e72ac6459db9e0896b
-  languageName: node
-  linkType: hard
-
 "console-browserify@npm:^1.1.0":
   version: 1.2.0
   resolution: "console-browserify@npm:1.2.0"
@@ -563,18 +460,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"debug@npm:^4":
-  version: 4.3.7
-  resolution: "debug@npm:4.3.7"
-  dependencies:
-    ms: "npm:^2.1.3"
-  peerDependenciesMeta:
-    supports-color:
-      optional: true
-  checksum: 10c0/1471db19c3b06d485a622d62f65947a19a23fbd0dd73f7fd3eafb697eec5360cde447fb075919987899b1a2096e85d35d4eb5a4de09a57600ac9cf7e6c8e768b
-  languageName: node
-  linkType: hard
-
 "define-data-property@npm:^1.0.1, define-data-property@npm:^1.1.4":
   version: 1.1.4
   resolution: "define-data-property@npm:1.1.4"
@@ -700,13 +585,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"escalade@npm:^3.1.1":
-  version: 3.2.0
-  resolution: "escalade@npm:3.2.0"
-  checksum: 10c0/ced4dd3a78e15897ed3be74e635110bbf3b08877b0a41be50dcb325ee0e0b5f65fc2d50e9845194d7c4633f327e2e1c6cce00a71b617c5673df0374201d67f65
-  languageName: node
-  linkType: hard
-
 "events@npm:^3.0.0":
   version: 3.3.0
   resolution: "events@npm:3.3.0"
@@ -732,15 +610,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"fill-range@npm:^7.1.1":
-  version: 7.1.1
-  resolution: "fill-range@npm:7.1.1"
-  dependencies:
-    to-regex-range: "npm:^5.0.1"
-  checksum: 10c0/b75b691bbe065472f38824f694c2f7449d7f5004aa950426a2c28f0306c60db9b880c0b0e4ed819997ffb882d1da02cfcfc819bddc94d71627f5269682edf018
-  languageName: node
-  linkType: hard
-
 "foreground-child@npm:^3.1.0":
   version: 3.1.1
   resolution: "foreground-child@npm:3.1.1"
@@ -769,25 +638,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"fsevents@npm:~2.3.2":
-  version: 2.3.3
-  resolution: "fsevents@npm:2.3.3"
-  dependencies:
-    node-gyp: "npm:latest"
-  checksum: 10c0/a1f0c44595123ed717febbc478aa952e47adfc28e2092be66b8ab1635147254ca6cfe1df792a8997f22716d4cbafc73309899ff7bfac2ac3ad8cf2e4ecc3ec60
-  conditions: os=darwin
-  languageName: node
-  linkType: hard
-
-"fsevents@patch:fsevents@npm%3A~2.3.2#optional!builtin<compat/fsevents>":
-  version: 2.3.3
-  resolution: "fsevents@patch:fsevents@npm%3A2.3.3#optional!builtin<compat/fsevents>::version=2.3.3&hash=df0bf1"
-  dependencies:
-    node-gyp: "npm:latest"
-  conditions: os=darwin
-  languageName: node
-  linkType: hard
-
 "function-bind@npm:^1.1.2":
   version: 1.1.2
   resolution: "function-bind@npm:1.1.2"
@@ -795,13 +645,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"get-caller-file@npm:^2.0.5":
-  version: 2.0.5
-  resolution: "get-caller-file@npm:2.0.5"
-  checksum: 10c0/c6c7b60271931fa752aeb92f2b47e355eac1af3a2673f47c9589e8f8a41adc74d45551c1bc57b5e66a80609f10ffb72b6f575e4370d61cc3f7f3aaff01757cde
-  languageName: node
-  linkType: hard
-
 "get-intrinsic@npm:^1.1.3, get-intrinsic@npm:^1.2.4":
   version: 1.2.4
   resolution: "get-intrinsic@npm:1.2.4"
@@ -815,15 +658,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"glob-parent@npm:~5.1.2":
-  version: 5.1.2
-  resolution: "glob-parent@npm:5.1.2"
-  dependencies:
-    is-glob: "npm:^4.0.1"
-  checksum: 10c0/cab87638e2112bee3f839ef5f6e0765057163d39c66be8ec1602f3823da4692297ad4e972de876ea17c44d652978638d2fd583c6713d0eb6591706825020c9ee
-  languageName: node
-  linkType: hard
-
 "glob@npm:^10.2.2, glob@npm:^10.3.10":
   version: 10.3.16
   resolution: "glob@npm:10.3.16"
@@ -855,20 +689,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"has-flag@npm:^3.0.0":
-  version: 3.0.0
-  resolution: "has-flag@npm:3.0.0"
-  checksum: 10c0/1c6c83b14b8b1b3c25b0727b8ba3e3b647f99e9e6e13eb7322107261de07a4c1be56fc0d45678fc376e09772a3a1642ccdaf8fc69bdf123b6c086598397ce473
-  languageName: node
-  linkType: hard
-
-"has-flag@npm:^4.0.0":
-  version: 4.0.0
-  resolution: "has-flag@npm:4.0.0"
-  checksum: 10c0/2e789c61b7888d66993e14e8331449e525ef42aac53c627cc53d1c3334e768bcb6abdc4f5f0de1478a25beec6f0bd62c7549058b7ac53e924040d4f301f02fd1
-  languageName: node
-  linkType: hard
-
 "has-property-descriptors@npm:^1.0.0, has-property-descriptors@npm:^1.0.2":
   version: 1.0.2
   resolution: "has-property-descriptors@npm:1.0.2"
@@ -993,13 +813,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"ignore-by-default@npm:^1.0.1":
-  version: 1.0.1
-  resolution: "ignore-by-default@npm:1.0.1"
-  checksum: 10c0/9ab6e70e80f7cc12735def7ecb5527cfa56ab4e1152cd64d294522827f2dcf1f6d85531241537dc3713544e88dd888f65cb3c49c7b2cddb9009087c75274e533
-  languageName: node
-  linkType: hard
-
 "imurmurhash@npm:^0.1.4":
   version: 0.1.4
   resolution: "imurmurhash@npm:0.1.4"
@@ -1038,22 +851,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"is-binary-path@npm:~2.1.0":
-  version: 2.1.0
-  resolution: "is-binary-path@npm:2.1.0"
-  dependencies:
-    binary-extensions: "npm:^2.0.0"
-  checksum: 10c0/a16eaee59ae2b315ba36fad5c5dcaf8e49c3e27318f8ab8fa3cdb8772bf559c8d1ba750a589c2ccb096113bb64497084361a25960899cb6172a6925ab6123d38
-  languageName: node
-  linkType: hard
-
-"is-extglob@npm:^2.1.1":
-  version: 2.1.1
-  resolution: "is-extglob@npm:2.1.1"
-  checksum: 10c0/5487da35691fbc339700bbb2730430b07777a3c21b9ebaecb3072512dfd7b4ba78ac2381a87e8d78d20ea08affb3f1971b4af629173a6bf435ff8a4c47747912
-  languageName: node
-  linkType: hard
-
 "is-fullwidth-code-point@npm:^3.0.0":
   version: 3.0.0
   resolution: "is-fullwidth-code-point@npm:3.0.0"
@@ -1061,15 +858,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"is-glob@npm:^4.0.1, is-glob@npm:~4.0.1":
-  version: 4.0.3
-  resolution: "is-glob@npm:4.0.3"
-  dependencies:
-    is-extglob: "npm:^2.1.1"
-  checksum: 10c0/17fb4014e22be3bbecea9b2e3a76e9e34ff645466be702f1693e8f1ee1adac84710d0be0bd9f967d6354036fd51ab7c2741d954d6e91dae6bb69714de92c197a
-  languageName: node
-  linkType: hard
-
 "is-lambda@npm:^1.0.1":
   version: 1.0.1
   resolution: "is-lambda@npm:1.0.1"
@@ -1077,13 +865,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"is-number@npm:^7.0.0":
-  version: 7.0.0
-  resolution: "is-number@npm:7.0.0"
-  checksum: 10c0/b4686d0d3053146095ccd45346461bc8e53b80aeb7671cc52a4de02dbbf7dc0d1d2a986e2fe4ae206984b4d34ef37e8b795ebc4f4295c978373e6575e295d811
-  languageName: node
-  linkType: hard
-
 "isarray@npm:^1.0.0, isarray@npm:~1.0.0":
   version: 1.0.0
   resolution: "isarray@npm:1.0.0"
@@ -1125,13 +906,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"lodash@npm:^4.17.21":
-  version: 4.17.21
-  resolution: "lodash@npm:4.17.21"
-  checksum: 10c0/d8cbea072bb08655bb4c989da418994b073a608dffa608b09ac04b43a791b12aeae7cd7ad919aa4c925f33b48490b5cfe6c1f71d827956071dae2e7bb3a6b74c
-  languageName: node
-  linkType: hard
-
 "lru-cache@npm:^10.0.1, lru-cache@npm:^10.2.0":
   version: 10.2.2
   resolution: "lru-cache@npm:10.2.2"
@@ -1203,15 +977,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"minimatch@npm:^3.1.2":
-  version: 3.1.2
-  resolution: "minimatch@npm:3.1.2"
-  dependencies:
-    brace-expansion: "npm:^1.1.7"
-  checksum: 10c0/0262810a8fc2e72cca45d6fd86bd349eee435eb95ac6aa45c9ea2180e7ee875ef44c32b55b5973ceabe95ea12682f6e3725cbb63d7a2d1da3ae1163c8b210311
-  languageName: node
-  linkType: hard
-
 "minimatch@npm:^9.0.1":
   version: 9.0.4
   resolution: "minimatch@npm:9.0.4"
@@ -1321,13 +1086,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"ms@npm:^2.1.3":
-  version: 2.1.3
-  resolution: "ms@npm:2.1.3"
-  checksum: 10c0/d924b57e7312b3b63ad21fc5b3dc0af5e78d61a1fc7cfb5457edaf26326bf62be5307cc87ffb6862ef1c2b33b0233cdb5d4f01c4c958cc0d660948b65a287a48
-  languageName: node
-  linkType: hard
-
 "negotiator@npm:^0.6.3":
   version: 0.6.3
   resolution: "negotiator@npm:0.6.3"
@@ -1386,26 +1144,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"nodemon@npm:^3.1.7":
-  version: 3.1.7
-  resolution: "nodemon@npm:3.1.7"
-  dependencies:
-    chokidar: "npm:^3.5.2"
-    debug: "npm:^4"
-    ignore-by-default: "npm:^1.0.1"
-    minimatch: "npm:^3.1.2"
-    pstree.remy: "npm:^1.1.8"
-    semver: "npm:^7.5.3"
-    simple-update-notifier: "npm:^2.0.0"
-    supports-color: "npm:^5.5.0"
-    touch: "npm:^3.1.0"
-    undefsafe: "npm:^2.0.5"
-  bin:
-    nodemon: bin/nodemon.js
-  checksum: 10c0/e0b46939abdbce251b1d6281005a5763cee57db295bb00bc4a753b0f5320dac00fe53547fb6764c70a086cf6d1238875cccb800fbc71544b3ecbd3ef71183c87
-  languageName: node
-  linkType: hard
-
 "nopt@npm:^7.0.0":
   version: 7.2.1
   resolution: "nopt@npm:7.2.1"
@@ -1417,13 +1155,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"normalize-path@npm:^3.0.0, normalize-path@npm:~3.0.0":
-  version: 3.0.0
-  resolution: "normalize-path@npm:3.0.0"
-  checksum: 10c0/e008c8142bcc335b5e38cf0d63cfd39d6cf2d97480af9abdbe9a439221fd4d749763bab492a8ee708ce7a194bb00c9da6d0a115018672310850489137b3da046
-  languageName: node
-  linkType: hard
-
 "object-inspect@npm:^1.13.1":
   version: 1.13.1
   resolution: "object-inspect@npm:1.13.1"
@@ -1524,13 +1255,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"picomatch@npm:^2.0.4, picomatch@npm:^2.2.1":
-  version: 2.3.1
-  resolution: "picomatch@npm:2.3.1"
-  checksum: 10c0/26c02b8d06f03206fc2ab8d16f19960f2ff9e81a658f831ecb656d8f17d9edc799e8364b1f4a7873e89d9702dff96204be0fa26fe4181f6843f040f819dac4be
-  languageName: node
-  linkType: hard
-
 "proc-log@npm:^3.0.0":
   version: 3.0.0
   resolution: "proc-log@npm:3.0.0"
@@ -1569,13 +1293,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"pstree.remy@npm:^1.1.8":
-  version: 1.1.8
-  resolution: "pstree.remy@npm:1.1.8"
-  checksum: 10c0/30f78c88ce6393cb3f7834216cb6e282eb83c92ccb227430d4590298ab2811bc4a4745f850a27c5178e79a8f3e316591de0fec87abc19da648c2b3c6eb766d14
-  languageName: node
-  linkType: hard
-
 "public-encrypt@npm:^4.0.0":
   version: 4.0.3
   resolution: "public-encrypt@npm:4.0.3"
@@ -1658,15 +1375,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"readdirp@npm:~3.6.0":
-  version: 3.6.0
-  resolution: "readdirp@npm:3.6.0"
-  dependencies:
-    picomatch: "npm:^2.2.1"
-  checksum: 10c0/6fa848cf63d1b82ab4e985f4cf72bd55b7dcfd8e0a376905804e48c3634b7e749170940ba77b32804d5fe93b3cc521aa95a8d7e7d725f830da6d93f3669ce66b
-  languageName: node
-  linkType: hard
-
 "readline-sync@npm:^1.4.7":
   version: 1.4.10
   resolution: "readline-sync@npm:1.4.10"
@@ -1674,13 +1382,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"require-directory@npm:^2.1.1":
-  version: 2.1.1
-  resolution: "require-directory@npm:2.1.1"
-  checksum: 10c0/83aa76a7bc1531f68d92c75a2ca2f54f1b01463cb566cf3fbc787d0de8be30c9dbc211d1d46be3497dac5785fe296f2dd11d531945ac29730643357978966e99
-  languageName: node
-  linkType: hard
-
 "retry@npm:^0.12.0":
   version: 0.12.0
   resolution: "retry@npm:0.12.0"
@@ -1698,15 +1399,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"rxjs@npm:^7.8.1":
-  version: 7.8.1
-  resolution: "rxjs@npm:7.8.1"
-  dependencies:
-    tslib: "npm:^2.1.0"
-  checksum: 10c0/3c49c1ecd66170b175c9cacf5cef67f8914dcbc7cd0162855538d365c83fea631167cacb644b3ce533b2ea0e9a4d0b12175186985f89d75abe73dbd8f7f06f68
-  languageName: node
-  linkType: hard
-
 "safe-buffer@npm:^5.0.1, safe-buffer@npm:^5.1.0, safe-buffer@npm:^5.1.1, safe-buffer@npm:^5.1.2, safe-buffer@npm:^5.2.0, safe-buffer@npm:^5.2.1, safe-buffer@npm:~5.2.0":
   version: 5.2.1
   resolution: "safe-buffer@npm:5.2.1"
@@ -1744,15 +1436,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"semver@npm:^7.5.3":
-  version: 7.6.3
-  resolution: "semver@npm:7.6.3"
-  bin:
-    semver: bin/semver.js
-  checksum: 10c0/88f33e148b210c153873cb08cfe1e281d518aaa9a666d4d148add6560db5cd3c582f3a08ccb91f38d5f379ead256da9931234ed122057f40bb5766e65e58adaf
-  languageName: node
-  linkType: hard
-
 "set-function-length@npm:^1.2.1":
   version: 1.2.2
   resolution: "set-function-length@npm:1.2.2"
@@ -1793,9 +1476,9 @@ __metadata:
   languageName: node
   linkType: hard
 
-"shadow-cljs@npm:2.28.18":
-  version: 2.28.18
-  resolution: "shadow-cljs@npm:2.28.18"
+"shadow-cljs@npm:2.28.11":
+  version: 2.28.11
+  resolution: "shadow-cljs@npm:2.28.11"
   dependencies:
     node-libs-browser: "npm:^2.2.1"
     readline-sync: "npm:^1.4.7"
@@ -1805,7 +1488,7 @@ __metadata:
     ws: "npm:^7.4.6"
   bin:
     shadow-cljs: cli/runner.js
-  checksum: 10c0/4732cd11a5722644a0a91ae5560a55f62432ae5317bd2d1fd5bf12af8354c81776f4fcfce5c477b43af1ac2ecd4a216887337e1b92cca37a1b8cb9c157a393c1
+  checksum: 10c0/c5c77d524ee8f44e4ae2ddc196af170d02405cc8731ea71f852c7b220fc1ba8aaf5cf33753fd8a7566c8749bb75d360f903dfb0d131bcdc6c2c33f44404bd6a3
   languageName: node
   linkType: hard
 
@@ -1825,13 +1508,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"shell-quote@npm:^1.8.1":
-  version: 1.8.1
-  resolution: "shell-quote@npm:1.8.1"
-  checksum: 10c0/8cec6fd827bad74d0a49347057d40dfea1e01f12a6123bf82c4649f3ef152fc2bc6d6176e6376bffcd205d9d0ccb4f1f9acae889384d20baff92186f01ea455a
-  languageName: node
-  linkType: hard
-
 "side-channel@npm:^1.0.6":
   version: 1.0.6
   resolution: "side-channel@npm:1.0.6"
@@ -1851,15 +1527,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"simple-update-notifier@npm:^2.0.0":
-  version: 2.0.0
-  resolution: "simple-update-notifier@npm:2.0.0"
-  dependencies:
-    semver: "npm:^7.5.3"
-  checksum: 10c0/2a00bd03bfbcbf8a737c47ab230d7920f8bfb92d1159d421bdd194479f6d01ebc995d13fbe13d45dace23066a78a3dc6642999b4e3b38b847e6664191575b20c
-  languageName: node
-  linkType: hard
-
 "smart-buffer@npm:^4.2.0":
   version: 4.2.0
   resolution: "smart-buffer@npm:4.2.0"
@@ -1960,7 +1627,7 @@ __metadata:
   languageName: node
   linkType: hard
 
-"string-width-cjs@npm:string-width@^4.2.0, string-width@npm:^4.1.0, string-width@npm:^4.2.0, string-width@npm:^4.2.3":
+"string-width-cjs@npm:string-width@^4.2.0, string-width@npm:^4.1.0":
   version: 4.2.3
   resolution: "string-width@npm:4.2.3"
   dependencies:
@@ -2018,33 +1685,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"supports-color@npm:^5.5.0":
-  version: 5.5.0
-  resolution: "supports-color@npm:5.5.0"
-  dependencies:
-    has-flag: "npm:^3.0.0"
-  checksum: 10c0/6ae5ff319bfbb021f8a86da8ea1f8db52fac8bd4d499492e30ec17095b58af11f0c55f8577390a749b1c4dde691b6a0315dab78f5f54c9b3d83f8fb5905c1c05
-  languageName: node
-  linkType: hard
-
-"supports-color@npm:^7.1.0":
-  version: 7.2.0
-  resolution: "supports-color@npm:7.2.0"
-  dependencies:
-    has-flag: "npm:^4.0.0"
-  checksum: 10c0/afb4c88521b8b136b5f5f95160c98dee7243dc79d5432db7efc27efb219385bbc7d9427398e43dd6cc730a0f87d5085ce1652af7efbe391327bc0a7d0f7fc124
-  languageName: node
-  linkType: hard
-
-"supports-color@npm:^8.1.1":
-  version: 8.1.1
-  resolution: "supports-color@npm:8.1.1"
-  dependencies:
-    has-flag: "npm:^4.0.0"
-  checksum: 10c0/ea1d3c275dd604c974670f63943ed9bd83623edc102430c05adb8efc56ba492746b6e95386e7831b872ec3807fd89dd8eb43f735195f37b5ec343e4234cc7e89
-  languageName: node
-  linkType: hard
-
 "tar@npm:^6.1.11, tar@npm:^6.1.2":
   version: 6.2.1
   resolution: "tar@npm:6.2.1"
@@ -2075,40 +1715,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"to-regex-range@npm:^5.0.1":
-  version: 5.0.1
-  resolution: "to-regex-range@npm:5.0.1"
-  dependencies:
-    is-number: "npm:^7.0.0"
-  checksum: 10c0/487988b0a19c654ff3e1961b87f471702e708fa8a8dd02a298ef16da7206692e8552a0250e8b3e8759270f62e9d8314616f6da274734d3b558b1fc7b7724e892
-  languageName: node
-  linkType: hard
-
-"touch@npm:^3.1.0":
-  version: 3.1.1
-  resolution: "touch@npm:3.1.1"
-  bin:
-    nodetouch: bin/nodetouch.js
-  checksum: 10c0/d2e4d269a42c846a22a29065b9af0b263de58effc85a1764bb7a2e8fc4b47700e9e2fcbd7eb1f5bffbb7c73d860f93600cef282b93ddac8f0b62321cb498b36e
-  languageName: node
-  linkType: hard
-
-"tree-kill@npm:^1.2.2":
-  version: 1.2.2
-  resolution: "tree-kill@npm:1.2.2"
-  bin:
-    tree-kill: cli.js
-  checksum: 10c0/7b1b7c7f17608a8f8d20a162e7957ac1ef6cd1636db1aba92f4e072dc31818c2ff0efac1e3d91064ede67ed5dc57c565420531a8134090a12ac10cf792ab14d2
-  languageName: node
-  linkType: hard
-
-"tslib@npm:^2.1.0":
-  version: 2.8.1
-  resolution: "tslib@npm:2.8.1"
-  checksum: 10c0/9c4759110a19c53f992d9aae23aac5ced636e99887b51b9e61def52611732872ff7668757d4e4c61f19691e36f4da981cd9485e869b4a7408d689f6bf1f14e62
-  languageName: node
-  linkType: hard
-
 "tty-browserify@npm:0.0.0":
   version: 0.0.0
   resolution: "tty-browserify@npm:0.0.0"
@@ -2116,13 +1722,6 @@ __metadata:
   languageName: node
   linkType: hard
 
-"undefsafe@npm:^2.0.5":
-  version: 2.0.5
-  resolution: "undefsafe@npm:2.0.5"
-  checksum: 10c0/96c0466a5fbf395917974a921d5d4eee67bca4b30d3a31ce7e621e0228c479cf893e783a109af6e14329b52fe2f0cb4108665fad2b87b0018c0df6ac771261d5
-  languageName: node
-  linkType: hard
-
 "unique-filename@npm:^3.0.0":
   version: 3.0.0
   resolution: "unique-filename@npm:3.0.0"
@@ -2216,7 +1815,7 @@ __metadata:
   languageName: node
   linkType: hard
 
-"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0, wrap-ansi@npm:^7.0.0":
+"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0":
   version: 7.0.0
   resolution: "wrap-ansi@npm:7.0.0"
   dependencies:
@@ -2275,38 +1874,9 @@ __metadata:
   languageName: node
   linkType: hard
 
-"y18n@npm:^5.0.5":
-  version: 5.0.8
-  resolution: "y18n@npm:5.0.8"
-  checksum: 10c0/4df2842c36e468590c3691c894bc9cdbac41f520566e76e24f59401ba7d8b4811eb1e34524d57e54bc6d864bcb66baab7ffd9ca42bf1eda596618f9162b91249
-  languageName: node
-  linkType: hard
-
 "yallist@npm:^4.0.0":
   version: 4.0.0
   resolution: "yallist@npm:4.0.0"
   checksum: 10c0/2286b5e8dbfe22204ab66e2ef5cc9bbb1e55dfc873bbe0d568aa943eb255d131890dfd5bf243637273d31119b870f49c18fcde2c6ffbb7a7a092b870dc90625a
   languageName: node
   linkType: hard
-
-"yargs-parser@npm:^21.1.1":
-  version: 21.1.1
-  resolution: "yargs-parser@npm:21.1.1"
-  checksum: 10c0/f84b5e48169479d2f402239c59f084cfd1c3acc197a05c59b98bab067452e6b3ea46d4dd8ba2985ba7b3d32a343d77df0debd6b343e5dae3da2aab2cdf5886b2
-  languageName: node
-  linkType: hard
-
-"yargs@npm:^17.7.2":
-  version: 17.7.2
-  resolution: "yargs@npm:17.7.2"
-  dependencies:
-    cliui: "npm:^8.0.1"
-    escalade: "npm:^3.1.1"
-    get-caller-file: "npm:^2.0.5"
-    require-directory: "npm:^2.1.1"
-    string-width: "npm:^4.2.3"
-    y18n: "npm:^5.0.5"
-    yargs-parser: "npm:^21.1.1"
-  checksum: 10c0/ccd7e723e61ad5965fffbb791366db689572b80cca80e0f96aad968dfff4156cd7cd1ad18607afe1046d8241e6fb2d6c08bf7fa7bfb5eaec818735d8feac8f05
-  languageName: node
-  linkType: hard

docker/devenv/docker-compose.yaml

@@ -43,6 +43,7 @@ services:
 
     environment:
       - EXTERNAL_UID=${CURRENT_USER_ID}
+      - PENPOT_SECRET_KEY=super-secret-devenv-key
       # SMTP setup
       - PENPOT_SMTP_ENABLED=true
       - PENPOT_SMTP_DEFAULT_FROM=no-reply@example.com

docker/devenv/files/nginx.conf

@@ -90,7 +90,6 @@ http {
             proxy_hide_header x-amz-meta-server-side-encryption;
             proxy_hide_header x-amz-server-side-encryption;
             proxy_pass $redirect_uri;
-            proxy_ssl_server_name on;
 
             add_header x-internal-redirect "$redirect_uri";
             add_header x-cache-control "$redirect_cache_control";

docker/images/docker-compose.yaml

@@ -29,15 +29,6 @@ x-flags: &penpot-flags
 x-uri: &penpot-public-uri
   PENPOT_PUBLIC_URI: http://localhost:9001
 
-x-body-size: &penpot-http-body-size
-  # Max body size (30MiB); Used for plain requests, should never be
-  # greater than multi-part size
-  PENPOT_HTTP_SERVER_MAX_BODY_SIZE: 31457280
-
-  # Max multipart body size (350MiB)
-  PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE: 367001600
-
-
 networks:
   penpot:
 
@@ -112,7 +103,7 @@ services:
       # - "traefik.http.routers.penpot-https.tls.certresolver=letsencrypt"
 
     environment:
-      << : [*penpot-flags, *penpot-http-body-size]
+      << : *penpot-flags
 
   penpot-backend:
     image: "penpotapp/backend:latest"
@@ -132,7 +123,7 @@ services:
     ## container.
 
     environment:
-      << : [*penpot-flags, *penpot-public-uri, *penpot-http-body-size]
+      << : [*penpot-flags, *penpot-public-uri]
 
       ## Penpot SECRET KEY. It serves as a master key from which other keys for subsystems
       ## (eg http sessions, or invitations) are derived.
@@ -210,7 +201,7 @@ services:
     environment:
       # Don't touch it; this uses an internal docker network to
       # communicate with the frontend.
-      PENPOT_PUBLIC_URI: http://penpot-frontend:8080
+      PENPOT_PUBLIC_URI: http://penpot-frontend
 
       ## Redis is used for the websockets notifications.
       PENPOT_REDIS_URI: redis://penpot-redis/0
@@ -270,3 +261,5 @@ services:
   #   ports:
   #     - 9000:9000
   #     - 9001:9001
+
+

docker/images/files/nginx-entrypoint.sh

@@ -22,9 +22,7 @@ update_flags /var/www/app/js/config.js
 export PENPOT_BACKEND_URI=${PENPOT_BACKEND_URI:-http://penpot-backend:6060};
 export PENPOT_EXPORTER_URI=${PENPOT_EXPORTER_URI:-http://penpot-exporter:6061};
 export PENPOT_INTERNAL_RESOLVER=${PENPOT_INTERNAL_RESOLVER:-127.0.0.11};
-export PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE=${PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE:-367001600}; # Default to 350MiB
 
-envsubst "\$PENPOT_BACKEND_URI,\$PENPOT_EXPORTER_URI,\$PENPOT_INTERNAL_RESOLVER,\$PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE" \
-         < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
+envsubst "\$PENPOT_BACKEND_URI,\$PENPOT_EXPORTER_URI,\$PENPOT_INTERNAL_RESOLVER" < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
 
 exec "$@";

docker/images/files/nginx.conf

@@ -64,7 +64,7 @@ http {
         listen 8080 default_server;
         server_name _;
 
-        client_max_body_size $PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE;
+        client_max_body_size 100M;
         charset utf-8;
 
         proxy_http_version 1.1;
@@ -92,7 +92,6 @@ http {
             proxy_hide_header x-amz-request-id;
             proxy_hide_header x-amz-meta-server-side-encryption;
             proxy_hide_header x-amz-server-side-encryption;
-            proxy_ssl_server_name on;
             proxy_pass $redirect_uri;
 
             add_header x-internal-redirect "$redirect_uri";

docs/plugins/create-a-plugin.md

@@ -7,14 +7,6 @@ title: 2. Create a Plugin
 
 This guide covers the creation of a Penpot plugin. Penpot offers two ways to kickstart your development:
 
-<p class="advice">
-Have you got an idea for a new plugin? Great! But first take a look at <a
-href="https://penpot.app/penpothub/plugins">the plugin overview</a> to see if already
-exists, and consider joining efforts with other developers. This does not imply that we
-won't accept plugins that do similar things, since anything can be improved and done in
-different ways.
-</p>
-
 1. Using a Template:
 
    - **Typescript template**: Using the <a target="_blank" href="https://github.com/penpot/penpot-plugin-starter-template">Penpot Plugin Starter Template</a>: A basic template with the required files for quickstarting your plugin. This template uses Typescript and Vite.

docs/plugins/deployment.md

@@ -216,9 +216,3 @@ Success! - Published to example-plugin-penpot.surge.sh
 ```
 
 5. Done!
-
-## 3.5. Submitting to Penpot
-
-To make your finished plugin available in our catalog, submit in on the [plugin submission page](https://penpot.app/penpothub/plugins/create-plugin). Once it becomes available any Penpot user will be able to install and use it.
-
-

docs/plugins/faq.md

@@ -65,7 +65,7 @@ You will be able to share your plugin with the <a target="_blank" href="https://
 
 ### My plugin works on my local machine, but I couldn’t install it on Penpot. What could be the problem?
 
-The url you that you need to provide in the plugin manager should look <a target="_blank" href="/plugins/create-a-plugin/#2.6.-step-6.-configure-the-manifest-file">like this</a>: <code class="language-bash">https:\/\/yourdomain.com/assets/manifest.json</code>
+The url you that you need to provide in the plugin manager should look <a target="_blank" href="/plugins/create-a-plugin/#2.6.-step-6.-configure-the-manifest-file">like this</a>: <code class="language-bash">https:\/\/yourdomain.com/assents/manifest.json</code>
 
 ### Where can I get support if I find a bug or an unexpected behavior?
 

docs/technical-guide/configuration.md

@@ -30,7 +30,7 @@ flags (that just enables or disables something). All flags are set in a single
 format: <code class="language-bash"><enable|disable>-\<flag-name></code>. For example:
 
 ```bash
-PENPOT_FLAGS: enable-smtp disable-registration disable-email-verification
+PENPOT_FLAGS: enable-smpt disable-registration disable-email-verification
 ```
 
 ### Registration ###
@@ -405,14 +405,6 @@ where users will access the application:
 PENPOT_PUBLIC_URI: http://localhost:9001
 ```
 
-<p class="advice">
-    If you plan to serve Penpot under different domain than `localhost` without HTTPS,
-    you need to disable the `secure` flag on cookies, with the `disable-secure-session-cookies` flag.
-    This is a configuration NOT recommended for production environments.
-</p>
-
-Check all the [flags](#other-flags) to fully customize your instance.
-
 ## Frontend ##
 
 In comparison with backend, frontend only has a small number of runtime configuration
@@ -432,8 +424,8 @@ To connect the frontend to the exporter and backend, you need to fill out these
 
 ```bash
 # Frontend
-PENPOT_BACKEND_URI: http://your-penpot-backend:6060
-PENPOT_EXPORTER_URI: http://your-penpot-exporter:6061
+PENPOT_BACKEND_URI: http://your-penpot-backend
+PENPOT_EXPORTER_URI: http://your-penpot-exporter
 ```
 
 These variables are used for generate correct nginx.conf file on container startup.
@@ -488,4 +480,3 @@ __Since version 2.0.0__
 [2]: /technical-guide/getting-started#configure-penpot-with-docker
 [3]: /technical-guide/developer/common#dev-environment
 [4]: https://github.com/penpot/penpot/blob/main/docker/images/files/nginx.conf
-

docs/technical-guide/getting-started.md

@@ -4,8 +4,7 @@ title: 1. Self-hosting Guide
 
 # Self-hosting Guide
 
-This guide explains how to get your own Penpot instance, running on a machine you control,
-to test it, use it by you or your team, or even customize and extend it any way you like.
+This guide explains how to get your own Penpot instance, running on a machine you control, to test it, use it by you or your team, or even customize and extend it any way you like.
 
 If you need more context you can look at the <a
 href="https://community.penpot.app/t/self-hosting-penpot-i/2336" target="_blank">post
@@ -15,30 +14,18 @@ about self-hosting</a> in Penpot community.
 href="https://design.penpot.app">our SaaS offer</a> for Penpot and your
 self-hosted Penpot platform!**
 
-There are three main options for creating a Penpot instance:
+There are two main options for creating a Penpot instance:
 
 1. Using the platform of our partner <a href="https://elest.io/open-source/penpot" target="_blank">Elestio</a>.
 2. Using <a href="https://docker.com" target="_blank">Docker</a> tool.
-3. Using <a href="https://kubernetes.io/" target="_blank">Kubernetes</a>.
 
 <p class="advice">
-The recommended way is to use Elestio, since it's simpler, fully automatic and still greatly flexible.
-Use Docker if you already know the tool, if need full control of the process or have extra requirements
-and do not want to depend on any external provider, or need to do any special customization.
+The recommended way is to use Elestio, since it's simpler, fully automatic and still greatly flexible. Use Docker if you already know the tool, if need full control of the process or have extra requirements and do not want to depend on any external provider, or need to do any special customization.
 </p>
 
 Or you can try <a href="#unofficial-self-host-options">other options</a>,
 offered by Penpot community.
 
-## Recommended settings
-To self-host Penpot, you’ll need a server with the following specifications:
-
-* **CPU:** 1-2 CPUs
-* **RAM:** 4 GiB of RAM
-* **Disk Space:** Disk requirements depend on your usage. Disk usage primarily involves the database and any files uploaded by users.
-
-This setup should be sufficient for a smooth experience with typical usage (your mileage may vary).
-
 ## Install with Elestio
 
 This section explains how to get Penpot up and running using <a href="https://elest.io/open-source/penpot"
@@ -227,9 +214,6 @@ docker compose -f docker-compose.yaml pull
 
 This will fetch the latest images. When you do <code class="language-bash">docker compose up</code> again, the containers will be recreated with the latest version.
 
-<p class="advice">
-    It is strongly recommended to update the Penpot version in small increments, rather than updating between two distant versions.
-</p>
 
 **Important: Upgrade from version 1.x to 2.0**
 
@@ -277,7 +261,7 @@ itself.
 
 This section details everything you need to know to get Penpot up and running in
 production environments using a Kubernetes cluster of your choice. To do this, we have
-created a <a href="https://helm.sh/" target="_blank">Helm</a> repository with everything
+created a <a href="https://helm.sh/" target="_blank">Helm<a> repository with everything
 you need.
 
 Therefore, your prerequisite will be to have a Kubernetes cluster on which we can install
@@ -303,7 +287,7 @@ in turn have its own release name.
 With these concepts in mind, we can now explain Helm like this:
 
 > Helm installs charts into Kubernetes clusters, creating a new release for each
-> installation. To find new charts, you can search Helm chart repositories.
+> installation. And to find new charts, you can search Helm chart repositories.
 
 
 ### Install Helm

docs/technical-guide/index.md

@@ -20,8 +20,6 @@ machine.
 
 * In the [Install with Docker][2] section, you can find the official Docker installation guide.
 
-* In the [Install with Kubernetes][7] section, you can find the official Kubernetes installation guide.
-
 * In the [Configuration][3] section, you can find all the customization options you can set up after installing.
 
 * Or you can try other, not supported by Penpot, [Unofficial options][4].
@@ -30,11 +28,9 @@ machine.
 
 The [Integration Guide][5] explains how to connect Penpot with external apps, so they get notified
 when certain events occur and may create your own interconnections and collaboration features.
-
 ## Developing Penpot
 
-Also, if you are a developer, you can get into the code, to explore it, learn how it is made,
-or extend it and contribute with new functionality. For this, we have a different Docker installation.
+Also, if you are a developer, you can get into the code, to explore it, learn how it is made, or extend it and contribute with new functionality. For this, we have a different Docker installation.
 In the [Developer Guide][6] you can find how to setup a development environment and many other dev-oriented documentation.
 
 [1]: /technical-guide/getting-started/#install-with-elestio
@@ -43,4 +39,3 @@ In the [Developer Guide][6] you can find how to setup a development environment
 [4]: /technical-guide/getting-started/#unofficial-self-host-options
 [5]: /technical-guide/integration/
 [6]: /technical-guide/developer/
-[7]: /technical-guide/getting-started/#install-with-kubernetes

docs/user-guide/import-export/index.njk

@@ -5,25 +5,45 @@ title: 14· Import/export files
 <h1 id="import-export">Import and export files</h1>
 <p class="main-paragraph">You can export Penpot files to your computer and import them from your computer to your projects.</p>
 
+<h2 id="penpot-formats">Penpot file formats</h2>
+<p>There are two different formats in which you can import/export Penpot files. A standard one and a binary one. You always have the chance to use both for any file.</p>
+<h3>Penpot file (.penpot).</h3>
+<p>The fast one. Binary Penpot specific.</p>
+<ul>
+  <li>✅ Highly efficient in terms of memory and transfer time when exporting and importing.</li>
+  <li>❌ It can be opened only in Penpot.</li>
+  <li>❌ Not transparent, code difficult to explore.</li>
+</ul>
+<h3>Standard file (.zip).</h3>
+<p>The open one. A compressed file that includes SVG and JSON.</p>
+<ul>
+  <li>✅ Allows the file to be opened by other softwares (still, for those cases export to SVG seems to be the common practice).</li>
+  <li>✅ Allows some automations and integrations.</li>
+  <li>✅ Is a transparent, existing, open standard format.</li>
+  <li>❌ Highly inefficient in terms of memory and transfer time when exporting and importing (this is because SVG).</li>
+</ul>
+
 <h2 id="files-export">Export Penpot files</h2>
 <p>Exporting files is useful for many reasons. Sometimes you want to have a backup of your files and sometimes it is useful to share Penpot files with a user that does not belong to one of your teams, or you want to have a backup of your files outside Penpot, both SaaS (design.penpot.app) or at a self-hosted instance.</p>
 
 <h3 id="export-penpot-files">How to export Penpot files</h3>
 <h4>Export a single file</h4>
 <p>You can download (export) files from the workspace and from the dashboard.</p>
-<p>
-  <strong>From the <a href="/user-guide/the-interface/#interface-dashboard">dashboard</a></strong>: Select the download option at the file card menu.
-  <figure><img src="/img/import-export/export-card.webp" alt="Export penpot file" /></figure>
-</p>
-<p>
-  <strong>From the <a href="/user-guide/the-interface/#interface-workspace">workspace</a></strong>: Select the download option at the main menu.
-  <figure><img src="/img/import-export/export-menu.webp" alt="Export penpot file" /></figure>
-</p>
+<ul>
+  <li>
+    <strong>From the <a href="/user-guide/the-interface/#interface-workspace">workspace</a></strong>: Select the download option at the main menu.
+    <figure><img src="/img/import-export/export-card.webp" alt="Export penpot file" /></figure>
+  </li>
+  <li>
+    <strong>From the <a href="/user-guide/the-interface/#interface-dashboard">dashboard</a></strong>: Select the download option at the file card menu.
+    <figure><img src="/img/import-export/export-menu.webp" alt="Export penpot file" /></figure>
+  </li>
+</ul>
 
 <h4>Export multiple files</h4>
 <p>Select multiple files to export them at the same time. An overlay will show you the progress of the different exports.</p>
 <figure>
-  <video title="Export multiple files" muted="" playsinline="" controls="" width="auto" poster="/img/import-export/export-multiple.webp" height="auto">
+  <video title="Export multiple files" muted="" playsinline="" controls="" width="100%" poster="/img/import-export/export-multiple.webp" height="auto">
     <source src="/img/import-export/export-multiple.mp4" type="video/mp4">
   </video>
 </figure>
@@ -43,27 +63,4 @@ title: 14· Import/export files
 <p>The import option is at the projects menu. Press “Import files” and then select one or more .penpot files to import. You can import a .zip file as well.</p>
 <figure><img src="/img/import-export/import-menu.webp" alt="Import penpot file" /></figure>
 <p>Right before importing the files to your project, you’ll still have the opportunity to review the items to be imported, have the information about the ones that can not be imported and also the chance to discard files.</p>
-<figure><img src="/img/import-export/import-selection.webp" alt="Import penpot file" /></figure>
-
-<h2 id="penpot-formats">Penpot file format</h2>
-<p>Penpot export to a unique format that streamline the import and export of files and assets by being more efficient and interoperable.</p>
-<p>Unlike other design tools, <strong>Penpot's format is built on standard languages</strong>. The exported file is essentially a ZIP archive containing binary assets (such as bitmap and vector images) alongside a readable JSON structure. By avoiding proprietary formats, Penpot empowers users with autonomy from specific tools while enabling seamless third-party integrations.</p>
-
-<h3>Deprecated Penpot file formats</h3>
-<p class="advice">These formats can only be exported from version 2.3 or earlier versions, but can be imported to any Penpot version</p>
-<p>There are two different deprecated Penpot file formats in which you can import/export Penpot files. A standard one and a binary one. You always have the chance to use both for any file.</p>
-<h4>[Deprecated] Penpot file (.penpot).</h4>
-<p>The fast one. Binary Penpot specific.</p>
-<ul>
-  <li>✅ Highly efficient in terms of memory and transfer time when exporting and importing.</li>
-  <li>❌ It can be opened only in Penpot.</li>
-  <li>❌ Not transparent, code difficult to explore.</li>
-</ul>
-<h4>[Deprecated] Standard file (.zip).</h4>
-<p>The open one. A compressed file that includes SVG and JSON.</p>
-<ul>
-  <li>✅ Allows the file to be opened by other softwares (still, for those cases export to SVG seems to be the common practice).</li>
-  <li>✅ Allows some automations and integrations.</li>
-  <li>✅ Is a transparent, existing, open standard format.</li>
-  <li>❌ Highly inefficient in terms of memory and transfer time when exporting and importing (this is because SVG).</li>
-</ul>
+<figure><img src="/img/import-export/import-selection.webp" alt="Import penpot file" /></figure

docs/user-guide/introduction/shortcuts.njk

@@ -424,6 +424,11 @@ title: Shortcuts
       <td style="text-align: center;"><kbd>Alt</kbd><kbd>P</kbd></td>
       <td style="text-align: center;"><kbd>⌥</kbd><kbd>P</kbd></td>
     </tr>
+    <tr>
+      <td>History</td>
+      <td style="text-align: center;"><kbd>Alt</kbd><kbd>H</kbd></td>
+      <td style="text-align: center;"><kbd>⌥</kbd><kbd>H</kbd></td>
+    </tr>
     <tr>
       <td>Layers</td>
       <td style="text-align: center;"><kbd>Alt</kbd><kbd>L</kbd></td>

docs/user-guide/styling/index.njk

@@ -124,10 +124,10 @@ title: 06· Styling
   <li><strong>Square</strong> - Adds a rectangular ending to the end of the path.</li>
 </ul>
 
-<h2 id="radius">Border radius</h2>
-<p>You can customize the border radius of rectangles and images, with the option to customize each corner individually.</p>
+<h2 id="radius">Corner radius</h2>
+<p>You can set values for corner radius to rectangles and images. There’s also the option to edit each corner individually.</p>
 <figure>
-  <video title="Border radius" muted="" playsinline="" controls="" width="100%" poster="/img/styling/corners.webp" height="auto">
+  <video title="Corner radius" muted="" playsinline="" controls="" width="100%" poster="/img/styling/corners.webp" height="auto">
     <source src="/img/styling/corners.mp4" type="video/mp4">
   </video>
 </figure>
@@ -155,30 +155,4 @@ title: 06· Styling
   <video title="Apply blur to a layer" muted="" playsinline="" controls="" width="100%" poster="/img/styling/blur.webp" height="auto">
     <source src="/img/styling/blur.mp4" type="video/mp4">
   </video>
-</figure>
-
-<h2 id="blend">Opacity and blend</h2>
-<p>Set the overal opacity for layers and their blend mode.</p> 
-<p>Blend allows you to control how a layer interacts with the layers beneath it, determining how pixels from the current layer are combined with pixels in the underlying layers. Use blend to achive various effects, such as shading, highlights, or creative visual styles.</p>
-<figure>
-  <img alt="Layer blend and opacity" src="/img/styling/blend-opacity.webp"/>
-</figure>
-<p>Blend options available:</p>
-<ul>
-  <li><strong>Normal</strong></li>
-  <li><strong>Darken</strong></li>
-  <li><strong>Multiply</strong></li>
-  <li><strong>Color burn</strong></li>
-  <li><strong>Lighten</strong></li>
-  <li><strong>Screen</strong></li>
-  <li><strong>Color dodge</strong></li>
-  <li><strong>Overlay</strong></li>
-  <li><strong>Soft light</strong></li>
-  <li><strong>Hard light</strong></li>
-  <li><strong>Difference</strong></li>
-  <li><strong>Exclusion</strong></li>
-  <li><strong>Hue</strong></li>
-  <li><strong>Saturation</strong></li>
-  <li><strong>Color</strong></li>
-  <li><strong>Luminosity</strong></li>
-</ul>
+</figure>

docs/user-guide/teams/index.njk

@@ -36,10 +36,9 @@ member is allowed to do depends on their permissions.</p>
 <h3>Team roles</h3>
 <p>These are the team roles currently available at Penpot:</p>
 <ul>
-  <li><strong>Viewer:</strong> Viewers can view, comment on and inspect files but will not be able to edit them, nor do they have permissions to manage team settings.</strong></li>
-  <li><strong>Editor:</strong> Editors can create, import, edit and manage files and libraries, but do not have permissions to manage team settings.</strong></li>
-  <li><strong>Admin:</strong> Admins have the same permissions as editors, with the added ability to change every other member's role except owners. They can invite members and update team settings.</strong></li>
-  <li><strong>Owner:</strong> There's only one owner per team, the role is automatically assigned to the team creator. Owners have all the permissions of admins, with the additional ability to change any member's role, including transferring ownership. Owners can update team settings, invite members and delete teams.</strong></li>
+  <li><strong>Owner:</strong> There's only one owner per team, the role is automatically assigned to the team creator. Owners have permissions to change every other member role, including transfering ownership. Owners can update team settings, invite members and delete teams.</strong></li>
+  <li><strong>Admin:</strong> Permissions to change every other member role except owners. Can invite members and update team settings.</strong></li>
+  <li><strong>Editor:</strong> Without permissions to change member roles, invite members or update team settings.</strong></li>
 </ul>
 <figure><img src="/img/teams/teams-permissions.webp" alt="Team members" /></figure>
 <p class="advice">More team roles will be eventually available, as well as fine grained permissions management to control members access and actions.</p>

docs/user-guide/workspace-basics/index.njk

@@ -199,58 +199,26 @@ geometric structure. In Penpot there are three types of guides:
   <img src="/img/workspace-basics/shortcuts.webp" alt="Shortcuts panel" />
 </figure>
 
-<h2 id="history">File history versions</h2>
-<p>The history panel keeps track of the latest changes on an opened file as well as the different versions of the file, making it easier to track changes, revert to previous states and collaborate.</p>
+<h2 id="history">History</h2>
+<p>The history panel keeps track of the latest changes on an opened file.</p>
 
-<h3>View history</h3>
-<p>To view the recent history of a file at the workspace click the history icon on the navbar at the left:</p>
-<ul>
-  <li>To see the history of file versions go to the <strong>History</strong> tab.</li>
-  <li>To see the history of item changes go to the <strong>Actions</strong> tab.</li>
-</ul>
+<h4>View history</h4>
+<p>To view the recent history of a file at the workspace press <kbd>Ctrl/⌘</kbd> + <kbd>H</kbd> or click at the history icon on the toolbar at the left.</p>
+<p>At the history you can see items with information about the last changes. At first sight you have object type (rectangle, text, image...) and type of change (New, Modified, Deleted...). If you press the item further details are shown.</p>
 <figure>
-  <img src="/img/workspace-basics/history-view.webp" alt="History versions button" />
+  <img src="/img/workspace-basics/history.webp" alt="History panel" />
 </figure>
+<p><strong>Note:</strong> History panel is still in a very early state and shows only a limited list of changes at a current browser tab session. Refreshing the browser means refreshing the History as well. Eventually, Penpot will have a proper version history capacity.</p>
 
-<h3>History panel</h3>
-<p>At the History panel, you can save the current version of your file, as well as access previous versions for up to 7 days.</p>
-
-<h4>Restore versions</h4>
-<p>All saved versions of the file—whether manually saved, autosaved, or pinned—can be restored, reverting the file back to its state at the selected time.</p>
+<h4>Navigate history</h4>
+<p>To navigate through the history press <kbd>Ctrl/⌘</kbd> + <kbd>Z</kbd> to go backwards and <kbd>Ctrl/⌘</kbd> + <kbd>Shift/⇧</kbd> + <kbd>Z</kbd> to go forward.</p>
+<p>You can also press any item of the history list to get to this specific state.</p>
 <figure>
-  <img src="/img/workspace-basics/history-restore.webp" alt="Restore versions" />
+  <video title="Navigate history" muted="" playsinline="" controls="" width="auto" poster="/img/workspace-basics/history-navigate.webp" height="auto">
+    <source src="/img/workspace-basics/history-navigate.mp4" type="video/mp4">
+  </video>
 </figure>
 
-<h4>Saved versions</h4>
-<p>You can save the current version of your file by clicking the pin icon at the History tab. This will allow the version to be named and it will add it to your list of versions.</p>
-<figure>
-  <img src="/img/workspace-basics/history-save.webp" alt="Saved versions" />
-</figure>
-
-<h4>Autosaved versions</h4>
-<p>When you start working on a file, Penpot will start to automatically save versions of that file across time so that you can later restore them as needed.</p>
-<p>In the History tab, if you click on the autosaved versions, you’ll see a list of the exact date and time when the version was automatically saved.</p>
-<figure>
-  <img src="/img/workspace-basics/history-autosaved.webp" alt="Autosaved versions" />
-</figure>
-
-<h4>Pinned versions</h4>
-<p>File versions can also be pinned. Pinning a file version will allow you to name it, making it easier to access at the History tab. Pinned file versions will be saved forever and can be renamed, restored or deleted at any time.</p>
-<figure>
-  <img src="/img/workspace-basics/history-pin.webp" alt="Pin versions" />
-</figure>
-
-<h3>Actions panel</h3>
-<p>At the Actions panel, you have the object type (rectangle, text, image...) and type of change (New, Modified, Deleted...). If you press the item, it will be reverted to its state before that specific action was performed.</p>
-<figure>
-  <img src="/img/workspace-basics/history-actions.webp" alt="Actions panel" />
-</figure>
-<p class="advice">The Actions panel shows only a limited list of changes at a current browser tab session. Refreshing the browser means refreshing the history of actions as well.</p>
-
-<h4>Navigate actions</h4>
-<p>To navigate through the actions press <kbd>Ctrl/⌘</kbd> + <kbd>Z</kbd> to go backwards and <kbd>Ctrl/⌘</kbd> + <kbd>Shift/⇧</kbd> + <kbd>Z</kbd> to go forward.</p>
-<p>You can also press any item of the actions list to get to this specific state.</p>
-
 <h2 id="comments">Comments</h2>
 <p>Comments allow the team to have one priceless conversation getting and providing feedback right over the designs and prototypes.<p>
 

frontend/deps.edn

@@ -43,9 +43,7 @@
   {:extra-paths ["dev"]
    :extra-deps
    {thheller/shadow-cljs {:mvn/version "2.28.18"}
-    com.bhauman/rebel-readline {:mvn/version "RELEASE"}
     org.clojure/tools.namespace {:mvn/version "RELEASE"}
-    criterium/criterium {:mvn/version "RELEASE"}
     cider/cider-nrepl {:mvn/version "0.48.0"}}}
 
   :shadow-cljs

frontend/dev/user.clj

@@ -1,34 +0,0 @@
-;; This Source Code Form is subject to the terms of the Mozilla Public
-;; License, v. 2.0. If a copy of the MPL was not distributed with this
-;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
-;;
-;; Copyright (c) KALEIDOS INC
-
-(ns user
-  (:require
-   [app.common.data :as d]
-   [app.common.pprint :as pp]
-   [clojure.java.io :as io]
-   [clojure.tools.namespace.repl :as repl]
-   [clojure.pprint :refer [pprint print-table]]
-   [clojure.repl :refer :all]
-   [clojure.walk :refer [macroexpand-all]]
-   [criterium.core  :as crit]))
-
-;; --- Benchmarking Tools
-
-(defmacro run-quick-bench
-  [& exprs]
-  `(crit/with-progress-reporting (crit/quick-bench (do ~@exprs) :verbose)))
-
-(defmacro run-quick-bench'
-  [& exprs]
-  `(crit/quick-bench (do ~@exprs)))
-
-(defmacro run-bench
-  [& exprs]
-  `(crit/with-progress-reporting (crit/bench (do ~@exprs) :verbose)))
-
-(defmacro run-bench'
-  [& exprs]
-  `(crit/bench (do ~@exprs)))

frontend/externs/worker.txt

@@ -1 +0,0 @@
-main.txt