Compare commits

..

32 Commits
2.4.2 ... 2.4.3

Author SHA1 Message Date
Andrey Antukh
10ca2b20e4 📎 Update changelog 2025-01-30 11:41:50 +01:00
Marina López
a2ce5efe69 Add has-files prop to create-file event 2025-01-30 11:31:45 +01:00
Andrey Antukh
15a896e050 🐛 Add migration for fix files with invalid token-data (#5712)
Because of previous bug that is already fixed
2025-01-30 09:17:02 +01:00
Aitor Moreno
8145eb89d7 🐛 Fix styles not being inherited (#5717) 2025-01-29 23:06:05 +01:00
Andrey Antukh
a6485b93b7 Merge pull request #5707 from penpot/niwinz-update-selfhost-defaults
 Add the ability to set http body size on docker images
2025-01-29 12:43:17 +01:00
Andrey Antukh
47f1ca9627 Change backend defaults for http body 2025-01-29 12:26:30 +01:00
Andrey Antukh
f252ffb201 Add the ability to overwrite default http body size on docker images
And provide a compose file with good defaults
2025-01-29 12:22:05 +01:00
Marina López
0768ef1b8f Add A/B test switching '+' to 'Add file' on hover (#5705) 2025-01-29 11:42:08 +01:00
Andrey Antukh
d9ba107da2 🔧 Update default body size for docker images
Set it to 350MiB, the same as we have on our saas
2025-01-29 11:38:16 +01:00
Alonso Torres
407b664910 🐛 Fix problem with plugin data null values (#5696) 2025-01-28 14:47:37 +01:00
Belén Albeza
31145f2805 Merge pull request #5675 from penpot/azazeln28-fix-issue-9900
🐛 Fix Colorpicker shows RGBA inputs when HSVA mode is active
2025-01-28 11:35:04 +01:00
AzazelN28
33192cfdb8 🐛 Fix colorpicker HSVA inputs 2025-01-27 15:38:21 +01:00
Andrey Antukh
471699960f 🐛 Update media references after instantiation of a component (#5652)
🐛 Update media references after instantiation of a component
2025-01-27 11:58:13 +01:00
Alonso Torres
7458a35f31 🐛 Fix problem when pasting text (#5671) 2025-01-24 11:26:06 +01:00
Pablo Alba
2ef22ecd08 🐛 Add migration fixing files with shape-ref cycles (#5663)
* 🐛 Add migration fixing files with shape-ref cycles

* :wip: Add optimized version of migration 62

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2025-01-23 17:28:13 +01:00
Andrey Antukh
9c60d1cdf9 Merge pull request #5660 from penpot/palba-consolidate-share-workspace
🎉 Consolidate add share button to the workspace
2025-01-23 13:06:16 +01:00
Andrey Antukh
080dc4b93c Merge remote-tracking branch 'origin/main' into staging 2025-01-23 12:42:04 +01:00
Pablo Alba
f0966070eb 🎉 Consolidate add share button to the workspace 2025-01-23 12:33:58 +01:00
Andrey Antukh
b1d053893c 📚 Add minor improvement to plugins creation documentation 2025-01-23 11:36:27 +01:00
Andrey Antukh
c2fb9f4c6f 📎 Add missing entry on the changelog file 2025-01-23 10:49:09 +01:00
Alejandro
19a26e46dc Merge pull request #5654 from penpot/niwinz-clone-template-bug
🐛 Add support for multiple file formats to clone-template
2025-01-23 09:01:39 +01:00
Andrey Antukh
efd4a11ae2 🐛 Add support for multiple formats on clone-template 2025-01-23 08:09:23 +01:00
Andrey Antukh
68bd8152b8 Merge pull request #5633 from penpot/eva-remove-tokens-from-measures
🐛  Fix errors from editable select on measures menu
2025-01-22 18:58:54 +01:00
Eva Marco
9e47a70adf 🐛 Fix errors from editable select on measures menu 2025-01-22 18:20:49 +01:00
Andrey Antukh
fae73a198c Merge remote-tracking branch 'origin/main' into staging 2025-01-22 17:44:24 +01:00
Andrey Antukh
6be1023c0a Merge tag '2.4.2' 2025-01-22 16:34:37 +01:00
Andrey Antukh
9bfee99672 Merge remote-tracking branch 'origin/main' into staging 2025-01-22 16:10:17 +01:00
Yamila Moreno
240f658c3a Merge pull request #5643 from penpot/yms-fix-docker-compose-configuration
🐳 fix docker compose documentation
2025-01-22 14:33:00 +01:00
Yamila Moreno
31bc7e7c86 🐳 add advice for unsecure configuration 2025-01-22 13:34:48 +01:00
Yamila Moreno
b3a5e6710f 🐳 improve docs about custom configuration 2025-01-22 12:21:13 +01:00
Andrey Antukh
85c1de4bda Merge pull request #5624 from penpot/yms-update-selfhosting-guide
🐳 improve docker documentation related to the updates
2025-01-20 16:36:40 +01:00
Yamila Moreno
d3ad15f19a 🐳 improve docker documentation related to the updates 2025-01-20 15:39:44 +01:00
39 changed files with 720 additions and 370 deletions

View File

@@ -1,5 +1,16 @@
# CHANGELOG
## 2.4.3
### :bug: Bugs fixed
- Fix errors from editable select on measures menu [Taiga #9888](https://tree.taiga.io/project/penpot/issue/9888)
- Fix exception on importing some templates from templates slider
- Consolidate adding share button to workspace
- Fix problem when pasting text [Taiga #9929](https://tree.taiga.io/project/penpot/issue/9929)
- Fix incorrect media reference handling on component instantiation
## 2.4.2
### :bug: Bugs fixed
@@ -28,12 +39,12 @@
(penpot). Because of that, the default NGINX listen port is now 8080 instead of 80, so
you will have to modify your infrastructure to apply this change.
- Redis 7.2 is explicitly pinned in our example docker-compose.yml file. This is done because,
starting with the next versions, Redis is no longer distributed under an open-source license.
On-premise users are obviously free to upgrade to the version they are using or a more modern one.
Keep in mind that if you were using a version other than 7.2, you may have to recreate the volume
associated with the Redis container because the 7.2 storage format may not be compatible with what
you already have stored on the volume, and Redis may not start. In the near future, we will evaluate
- Redis 7.2 is explicitly pinned in our example docker-compose.yml file. This is done because,
starting with the next versions, Redis is no longer distributed under an open-source license.
On-premise users are obviously free to upgrade to the version they are using or a more modern one.
Keep in mind that if you were using a version other than 7.2, you may have to recreate the volume
associated with the Redis container because the 7.2 storage format may not be compatible with what
you already have stored on the volume, and Redis may not start. In the near future, we will evaluate
whether to move to an open-source version of Redis (such as https://valkey.io/).
### :heart: Community contributions (Thank you!)

View File

@@ -114,37 +114,13 @@ Debug Main Page
</fieldset>
<fieldset>
<legend>Import binfile:</legend>
<desc>Import penpot file in binary
format. If <strong>overwrite</strong> is checked, all files will
be overwritten using the same ids found in the file instead of
generating a new ones.</desc>
<desc>Import penpot file in binary format.</desc>
<form method="post" enctype="multipart/form-data" action="/dbg/file/import">
<div class="row">
<input type="file" name="file" value="" />
</div>
<div class="row">
<label>Overwrite?</label>
<input type="checkbox" name="overwrite" />
<br />
<small>
Instead of creating a new file with all relations remapped,
reuses all ids and updates/overwrites the objects that are
already exists on the database.
<strong>Warning, this operation should be used with caution.</strong>
</small>
</div>
<div class="row">
<label>Migrate?</label>
<input type="checkbox" name="migrate" />
<br />
<small>
Applies the file migrations on the importation process.
</small>
</div>
<div class="row">
<input type="submit" name="upload" value="Upload" />
</div>

View File

@@ -12,6 +12,7 @@
[app.common.data.macros :as dm]
[app.common.exceptions :as ex]
[app.common.features :as cfeat]
[app.common.files.helpers :as cfh]
[app.common.files.migrations :as fmg]
[app.common.files.validate :as fval]
[app.common.logging :as l]
@@ -29,8 +30,9 @@
[app.util.time :as dt]
[app.worker :as-alias wrk]
[clojure.set :as set]
[clojure.walk :as walk]
[cuerdas.core :as str]))
[cuerdas.core :as str]
[datoteka.fs :as fs]
[datoteka.io :as io]))
(set! *warn-on-reflection* true)
@@ -52,6 +54,20 @@
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn parse-file-format
[template]
(assert (fs/path? template) "expected InputStream for `template`")
(with-open [^java.lang.AutoCloseable input (io/input-stream template)]
(let [buffer (byte-array 4)]
(io/read-to-buffer input buffer)
(if (and (= (aget buffer 0) 80)
(= (aget buffer 1) 75)
(= (aget buffer 2) 3)
(= (aget buffer 3) 4))
:binfile-v3
:binfile-v1))))
(def xf-map-id
(map :id))
@@ -225,40 +241,65 @@
:data nil}
{::sql/columns [:media-id :file-id :revn]}))
(def ^:private sql:get-missing-media-references
"SELECT fmo.*
FROM file_media_object AS fmo
WHERE fmo.id = ANY(?::uuid[])
AND file_id != ?")
(def ^:private
xform:collect-media-id
(comp
(map :objects)
(mapcat vals)
(mapcat (fn [obj]
;; NOTE: because of some bug, we ended with
;; many shape types having the ability to
;; have fill-image attribute (which initially
;; designed for :path shapes).
(sequence
(keep :id)
(concat [(:fill-image obj)
(:metadata obj)]
(map :fill-image (:fills obj))
(map :stroke-image (:strokes obj))
(->> (:content obj)
(tree-seq map? :children)
(mapcat :fills)
(map :fill-image))))))))
(defn update-media-references!
"Given a file and a coll of media-refs, check if all provided
references are correct or fix them in-place"
[{:keys [::db/conn] :as cfg} {file-id :id :as file} media-refs]
(let [missing-index
(reduce (fn [result {:keys [id] :as fmo}]
(assoc result id
(-> fmo
(assoc :id (uuid/next))
(assoc :file-id file-id)
(dissoc :created-at)
(dissoc :deleted-at))))
{}
(db/exec! conn [sql:get-missing-media-references
(->> (into #{} xf-map-id media-refs)
(db/create-array conn "uuid"))
file-id]))
(defn collect-used-media
"Given a fdata (file data), returns all media references."
[data]
(-> #{}
(into xform:collect-media-id (vals (:pages-index data)))
(into xform:collect-media-id (vals (:components data)))
(into (keys (:media data)))))
lookup-index
(fn [id]
(if-let [mobj (get missing-index id)]
(do
(l/trc :hint "lookup index"
:file-id (str file-id)
:snap-id (str (:snapshot-id file))
:id (str id)
:result (str (get mobj :id)))
(get mobj :id))
id))
update-shapes
(fn [data {:keys [page-id shape-id]}]
(d/update-in-when data [:pages-index page-id :objects shape-id] cfh/relink-media-refs lookup-index))
file
(update file :data #(reduce update-shapes % media-refs))]
(doseq [[old-id item] missing-index]
(l/dbg :hint "create missing references"
:file-id (str file-id)
:snap-id (str (:snapshot-id file))
:old-id (str old-id)
:id (str (:id item)))
(db/insert! conn :file-media-object item
{::db/return-keys false}))
file))
(defn get-file-media
[cfg {:keys [data id] :as file}]
(db/run! cfg (fn [{:keys [::db/conn]}]
(let [ids (collect-used-media data)
(let [ids (cfh/collect-used-media data)
ids (db/create-array conn "uuid" ids)
sql (str "SELECT * FROM file_media_object WHERE id = ANY(?)")]
@@ -311,48 +352,7 @@
replace the old :component-file reference with the new
ones, using the provided file-index."
[data]
(letfn [(process-map-form [form]
(cond-> form
;; Relink image shapes
(and (map? (:metadata form))
(= :image (:type form)))
(update-in [:metadata :id] lookup-index)
;; Relink paths with fill image
(map? (:fill-image form))
(update-in [:fill-image :id] lookup-index)
;; This covers old shapes and the new :fills.
(uuid? (:fill-color-ref-file form))
(update :fill-color-ref-file lookup-index)
;; This covers the old shapes and the new :strokes
(uuid? (:stroke-color-ref-file form))
(update :stroke-color-ref-file lookup-index)
;; This covers all text shapes that have typography referenced
(uuid? (:typography-ref-file form))
(update :typography-ref-file lookup-index)
;; This covers the component instance links
(uuid? (:component-file form))
(update :component-file lookup-index)
;; This covers the shadows and grids (they have directly
;; the :file-id prop)
(uuid? (:file-id form))
(update :file-id lookup-index)))
(process-form [form]
(if (map? form)
(try
(process-map-form form)
(catch Throwable cause
(l/warn :hint "failed form" :form (pr-str form) ::l/sync? true)
(throw cause)))
form))]
(walk/postwalk process-form data)))
(cfh/relink-media-refs data lookup-index))
(defn- relink-media
"A function responsible of process the :media attr of file data and

View File

@@ -298,7 +298,7 @@
(defmulti write-section ::section)
(defn write-export!
[{:keys [::include-libraries ::embed-assets] :as cfg}]
[{:keys [::bfc/include-libraries ::bfc/embed-assets] :as cfg}]
(when (and include-libraries embed-assets)
(throw (IllegalArgumentException.
"the `include-libraries` and `embed-assets` are mutally excluding options")))
@@ -323,7 +323,7 @@
[:v1/metadata :v1/files :v1/rels :v1/sobjects]))))
(defmethod write-section :v1/metadata
[{:keys [::output ::ids ::include-libraries] :as cfg}]
[{:keys [::output ::bfc/ids ::bfc/include-libraries] :as cfg}]
(if-let [fids (get-files cfg ids)]
(let [lids (when include-libraries
(bfc/get-libraries cfg ids))
@@ -335,7 +335,7 @@
:hint "unable to retrieve files for export")))
(defmethod write-section :v1/files
[{:keys [::output ::embed-assets ::include-libraries] :as cfg}]
[{:keys [::output ::bfc/embed-assets ::bfc/include-libraries] :as cfg}]
;; Initialize SIDS with empty vector
(vswap! bfc/*state* assoc :sids [])
@@ -382,7 +382,7 @@
(vswap! bfc/*state* update :sids into bfc/xf-map-media-id thumbnails))))
(defmethod write-section :v1/rels
[{:keys [::output ::include-libraries] :as cfg}]
[{:keys [::output ::bfc/include-libraries] :as cfg}]
(let [ids (-> bfc/*state* deref :files set)
rels (when include-libraries
(bfc/get-files-rels cfg ids))]
@@ -421,15 +421,15 @@
(defmulti read-import ::version)
(defmulti read-section ::section)
(s/def ::profile-id ::us/uuid)
(s/def ::project-id ::us/uuid)
(s/def ::input io/input-stream?)
(s/def ::bfc/profile-id ::us/uuid)
(s/def ::bfc/project-id ::us/uuid)
(s/def ::bfc/input io/input-stream?)
(s/def ::overwrite? (s/nilable ::us/boolean))
(s/def ::ignore-index-errors? (s/nilable ::us/boolean))
;; FIXME: replace with schema
(s/def ::read-import-options
(s/keys :req [::db/pool ::sto/storage ::project-id ::profile-id ::input]
(s/keys :req [::db/pool ::sto/storage ::bfc/project-id ::bfc/profile-id ::bfc/input]
:opt [::overwrite? ::ignore-index-errors?]))
(defn read-import!
@@ -439,7 +439,7 @@
`::bfc/overwrite`: if true, instead of creating new files and remapping id references,
it reuses all ids and updates existing objects; defaults to `false`."
[{:keys [::input ::bfc/timestamp] :or {timestamp (dt/now)} :as options}]
[{:keys [::bfc/input ::bfc/timestamp] :or {timestamp (dt/now)} :as options}]
(dm/assert!
"expected input stream"
@@ -453,7 +453,7 @@
(read-import (assoc options ::version version ::bfc/timestamp timestamp))))
(defn- read-import-v1
[{:keys [::db/conn ::project-id ::profile-id ::input] :as cfg}]
[{:keys [::db/conn ::bfc/project-id ::bfc/profile-id ::bfc/input] :as cfg}]
(bfc/disable-database-timeouts! cfg)
@@ -473,7 +473,7 @@
(let [options (-> cfg
(assoc ::bfc/features features)
(assoc ::section section)
(assoc ::input input))]
(assoc ::bfc/input input))]
(binding [bfc/*options* options]
(events/tap :progress {:op :import :section section})
(read-section options))))
@@ -491,7 +491,7 @@
(db/tx-run! options read-import-v1))
(defmethod read-section :v1/metadata
[{:keys [::input]}]
[{:keys [::bfc/input]}]
(let [{:keys [version files]} (read-obj! input)]
(l/dbg :hint "metadata readed"
:version (:full version)
@@ -509,7 +509,7 @@
thumbnails))
(defmethod read-section :v1/files
[{:keys [::db/conn ::input ::project-id ::bfc/overwrite ::name] :as system}]
[{:keys [::db/conn ::bfc/input ::bfc/project-id ::bfc/overwrite ::bfc/name] :as system}]
(doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))]
(let [file (read-obj! input)
@@ -576,7 +576,7 @@
file-id'))))
(defmethod read-section :v1/rels
[{:keys [::db/conn ::input ::bfc/timestamp]}]
[{:keys [::db/conn ::bfc/input ::bfc/timestamp]}]
(let [rels (read-obj! input)
ids (into #{} (-> bfc/*state* deref :files))]
;; Insert all file relations
@@ -600,7 +600,7 @@
::l/sync? true))))))
(defmethod read-section :v1/sobjects
[{:keys [::db/conn ::input ::bfc/overwrite ::bfc/timestamp] :as cfg}]
[{:keys [::db/conn ::bfc/input ::bfc/overwrite ::bfc/timestamp] :as cfg}]
(let [storage (sto/resolve cfg)
ids (read-obj! input)
thumb? (into #{} (map :media-id) (:thumbnails @bfc/*state*))]
@@ -674,17 +674,17 @@
"Do the exportation of a specified file in custom penpot binary
format. There are some options available for customize the output:
`::include-libraries`: additionally to the specified file, all the
`::bfc/include-libraries`: additionally to the specified file, all the
linked libraries also will be included (including transitive
dependencies).
`::embed-assets`: instead of including the libraries, embed in the
`::bfc/embed-assets`: instead of including the libraries, embed in the
same file library all assets used from external libraries."
[{:keys [::ids] :as cfg} output]
[{:keys [::bfc/ids] :as cfg} output]
(dm/assert!
"expected a set of uuid's for `::ids` parameter"
"expected a set of uuid's for `::bfc/ids` parameter"
(and (set? ids)
(every? uuid? ids)))
@@ -719,12 +719,12 @@
:cause @cs)))))
(defn import-files!
[{:keys [::input] :as cfg}]
[{:keys [::bfc/input] :as cfg}]
(dm/assert!
"expected valid profile-id and project-id on `cfg`"
(and (uuid? (::profile-id cfg))
(uuid? (::project-id cfg))))
(and (uuid? (::bfc/profile-id cfg))
(uuid? (::bfc/project-id cfg))))
(dm/assert!
"expected instance of jio/IOFactory for `input`"
@@ -738,7 +738,7 @@
(try
(binding [*position* (atom 0)]
(pu/with-open [input (io/input-stream input)]
(read-import! (assoc cfg ::input input))))
(read-import! (assoc cfg ::bfc/input input))))
(catch ZstdIOException cause
(ex/raise :type :validation

View File

@@ -192,7 +192,7 @@
(.closeEntry output))
(defn- get-file
[{:keys [::embed-assets ::include-libraries] :as cfg} file-id]
[{:keys [::bfc/embed-assets ::bfc/include-libraries] :as cfg} file-id]
(when (and include-libraries embed-assets)
(throw (IllegalArgumentException.
@@ -330,7 +330,7 @@
(write-entry! output path color)))))
(defn- export-files
[{:keys [::ids ::include-libraries ::output] :as cfg}]
[{:keys [::bfc/ids ::bfc/include-libraries ::output] :as cfg}]
(let [ids (into ids (when include-libraries (bfc/get-libraries cfg ids)))
rels (if include-libraries
(->> (bfc/get-files-rels cfg ids)
@@ -509,7 +509,7 @@
(json/read reader :key-fn json/read-kebab-key)))
(defn- read-file
[{:keys [::input ::file-id]}]
[{:keys [::bfc/input ::file-id]}]
(let [path (str "files/" file-id ".json")
entry (get-zip-entry input path)]
(-> (read-entry input entry)
@@ -517,7 +517,7 @@
(validate-file))))
(defn- read-file-plugin-data
[{:keys [::input ::file-id]}]
[{:keys [::bfc/input ::file-id]}]
(let [path (str "files/" file-id "/plugin-data.json")
entry (get-zip-entry* input path)]
(some->> entry
@@ -526,7 +526,7 @@
(validate-plugin-data))))
(defn- read-file-media
[{:keys [::input ::file-id ::entries]}]
[{:keys [::bfc/input ::file-id ::entries]}]
(->> (keep (match-media-entry-fn file-id) entries)
(reduce (fn [result {:keys [id entry]}]
(let [object (->> (read-entry input entry)
@@ -540,7 +540,7 @@
(not-empty)))
(defn- read-file-colors
[{:keys [::input ::file-id ::entries]}]
[{:keys [::bfc/input ::file-id ::entries]}]
(->> (keep (match-color-entry-fn file-id) entries)
(reduce (fn [result {:keys [id entry]}]
(let [object (->> (read-entry input entry)
@@ -553,7 +553,7 @@
(not-empty)))
(defn- read-file-components
[{:keys [::input ::file-id ::entries]}]
[{:keys [::bfc/input ::file-id ::entries]}]
(->> (keep (match-component-entry-fn file-id) entries)
(reduce (fn [result {:keys [id entry]}]
(let [object (->> (read-entry input entry)
@@ -566,7 +566,7 @@
(not-empty)))
(defn- read-file-typographies
[{:keys [::input ::file-id ::entries]}]
[{:keys [::bfc/input ::file-id ::entries]}]
(->> (keep (match-typography-entry-fn file-id) entries)
(reduce (fn [result {:keys [id entry]}]
(let [object (->> (read-entry input entry)
@@ -579,7 +579,7 @@
(not-empty)))
(defn- read-file-shapes
[{:keys [::input ::file-id ::page-id ::entries] :as cfg}]
[{:keys [::bfc/input ::file-id ::page-id ::entries] :as cfg}]
(->> (keep (match-shape-entry-fn file-id page-id) entries)
(reduce (fn [result {:keys [id entry]}]
(let [object (->> (read-entry input entry)
@@ -592,7 +592,7 @@
(not-empty)))
(defn- read-file-pages
[{:keys [::input ::file-id ::entries] :as cfg}]
[{:keys [::bfc/input ::file-id ::entries] :as cfg}]
(->> (keep (match-page-entry-fn file-id) entries)
(keep (fn [{:keys [id entry]}]
(let [page (->> (read-entry input entry)
@@ -608,7 +608,7 @@
(d/ordered-map))))
(defn- read-file-thumbnails
[{:keys [::input ::file-id ::entries] :as cfg}]
[{:keys [::bfc/input ::file-id ::entries] :as cfg}]
(->> (keep (match-thumbnail-entry-fn file-id) entries)
(reduce (fn [result {:keys [page-id frame-id tag entry]}]
(let [object (->> (read-entry input entry)
@@ -638,7 +638,7 @@
:plugin-data plugin-data}))
(defn- import-file
[{:keys [::db/conn ::project-id ::file-id ::file-name] :as cfg}]
[{:keys [::db/conn ::bfc/project-id ::file-id ::file-name] :as cfg}]
(let [file-id' (bfc/lookup-index file-id)
file (read-file cfg)
media (read-file-media cfg)
@@ -714,7 +714,7 @@
:library-file-id libr-id})))))
(defn- import-storage-objects
[{:keys [::input ::entries ::bfc/timestamp] :as cfg}]
[{:keys [::bfc/input ::entries ::bfc/timestamp] :as cfg}]
(events/tap :progress {:section :storage-objects})
(let [storage (sto/resolve cfg)
@@ -810,7 +810,7 @@
{::db/on-conflict-do-nothing? (::bfc/overwrite cfg)}))))
(defn- import-files
[{:keys [::bfc/timestamp ::input ::name] :or {timestamp (dt/now)} :as cfg}]
[{:keys [::bfc/timestamp ::bfc/input ::bfc/name] :or {timestamp (dt/now)} :as cfg}]
(dm/assert!
"expected zip file"
@@ -878,17 +878,17 @@
"Do the exportation of a specified file in custom penpot binary
format. There are some options available for customize the output:
`::include-libraries`: additionally to the specified file, all the
`::bfc/include-libraries`: additionally to the specified file, all the
linked libraries also will be included (including transitive
dependencies).
`::embed-assets`: instead of including the libraries, embed in the
`::bfc/embed-assets`: instead of including the libraries, embed in the
same file library all assets used from external libraries."
[{:keys [::ids] :as cfg} output]
[{:keys [::bfc/ids] :as cfg} output]
(dm/assert!
"expected a set of uuid's for `::ids` parameter"
"expected a set of uuid's for `::bfc/ids` parameter"
(and (set? ids)
(every? uuid? ids)))
@@ -930,14 +930,13 @@
:aborted @ab
:cause @cs)))))
(defn import-files!
[{:keys [::input] :as cfg}]
[{:keys [::bfc/input] :as cfg}]
(dm/assert!
"expected valid profile-id and project-id on `cfg`"
(and (uuid? (::profile-id cfg))
(uuid? (::project-id cfg))))
(and (uuid? (::bfc/profile-id cfg))
(uuid? (::bfc/project-id cfg))))
(dm/assert!
"expected instance of jio/IOFactory for `input`"
@@ -950,7 +949,7 @@
(l/info :hint "import: started" :id (str id))
(try
(with-open [input (ZipFile. (fs/file input))]
(import-files (assoc cfg ::input input)))
(import-files (assoc cfg ::bfc/input input)))
(catch Throwable cause
(vreset! cs cause)

View File

@@ -42,8 +42,8 @@
(def default-params
{::port 6060
::host "0.0.0.0"
::max-body-size (* 1024 1024 30) ; default 30 MiB
::max-multipart-body-size (* 1024 1024 120)}) ; default 120 MiB
::max-body-size 31457280 ; default 30 MiB
::max-multipart-body-size 367001600}) ; default 350 MiB
(defmethod ig/expand-key ::server
[k v]

View File

@@ -7,7 +7,9 @@
(ns app.http.debug
(:refer-clojure :exclude [error-handler])
(:require
[app.binfile.common :as bfc]
[app.binfile.v1 :as bf.v1]
[app.binfile.v3 :as bf.v3]
[app.common.data :as d]
[app.common.exceptions :as ex]
[app.common.logging :as l]
@@ -280,23 +282,23 @@
(ex/raise :type :validation
:code :missing-arguments))
(let [path (tmp/tempfile :prefix "penpot.export.")]
(let [path (tmp/tempfile :prefix "penpot.export." :min-age "30m")]
(with-open [output (io/output-stream path)]
(-> cfg
(assoc ::bf.v1/ids file-ids)
(assoc ::bf.v1/embed-assets embed?)
(assoc ::bf.v1/include-libraries libs?)
(bf.v1/export-files! output)))
(assoc ::bfc/ids file-ids)
(assoc ::bfc/embed-assets embed?)
(assoc ::bfc/include-libraries libs?)
(bf.v3/export-files! output)))
(if clone?
(let [profile (profile/get-profile pool profile-id)
project-id (:default-project-id profile)
cfg (assoc cfg
::bf.v1/overwrite false
::bf.v1/profile-id profile-id
::bf.v1/project-id project-id
::bf.v1/input path)]
(bf.v1/import-files! cfg)
::bfc/overwrite false
::bfc/profile-id profile-id
::bfc/project-id project-id
::bfc/input path)]
(bf.v3/import-files! cfg)
{::yres/status 200
::yres/headers {"content-type" "text/plain"}
::yres/body "OK CLONED"})
@@ -315,23 +317,24 @@
:hint "missing upload file"))
(let [profile (profile/get-profile pool profile-id)
project-id (:default-project-id profile)
overwrite? (contains? params :overwrite)
migrate? (contains? params :migrate)]
project-id (:default-project-id profile)]
(when-not project-id
(ex/raise :type :validation
:code :missing-project
:hint "project not found"))
(let [path (-> params :file :path)
cfg (assoc cfg
::bf.v1/overwrite overwrite?
::bf.v1/migrate migrate?
::bf.v1/profile-id profile-id
::bf.v1/project-id project-id
::bf.v1/input path)]
(bf.v1/import-files! cfg)
(let [path (-> params :file :path)
format (bfc/parse-file-format path)
cfg (assoc cfg
::bfc/profile-id profile-id
::bfc/project-id project-id
::bfc/input path)]
(if (= format :binfile-v3)
(bf.v3/import-files! cfg)
(bf.v1/import-files! cfg))
{::yres/status 200
::yres/headers {"content-type" "text/plain"}
::yres/body "OK"})))

View File

@@ -64,7 +64,8 @@
(catch Throwable cause
(events/tap :error (errors/handle' cause request))
(when-not (ex/instance? java.io.EOFException cause)
(l/err :hint "unexpected error on processing sse response" :cause cause)))
(binding [l/*context* (errors/request->context request)]
(l/err :hint "unexpected error on processing sse response" :cause cause))))
(finally
(sp/close! events/*channel*)
(px/await! listener)))))))}))

View File

@@ -7,6 +7,7 @@
(ns app.rpc.commands.binfile
(:refer-clojure :exclude [assert])
(:require
[app.binfile.common :as bfc]
[app.binfile.v1 :as bf.v1]
[app.binfile.v3 :as bf.v3]
[app.common.logging :as l]
@@ -46,9 +47,9 @@
(fn [_ output-stream]
(try
(-> cfg
(assoc ::bf.v1/ids #{file-id})
(assoc ::bf.v1/embed-assets embed-assets)
(assoc ::bf.v1/include-libraries include-libraries)
(assoc ::bfc/ids #{file-id})
(assoc ::bfc/embed-assets embed-assets)
(assoc ::bfc/include-libraries include-libraries)
(bf.v1/export-files! output-stream))
(catch Throwable cause
(l/err :hint "exception on exporting file"
@@ -61,9 +62,9 @@
(fn [_ output-stream]
(try
(-> cfg
(assoc ::bf.v3/ids #{file-id})
(assoc ::bf.v3/embed-assets embed-assets)
(assoc ::bf.v3/include-libraries include-libraries)
(assoc ::bfc/ids #{file-id})
(assoc ::bfc/embed-assets embed-assets)
(assoc ::bfc/include-libraries include-libraries)
(bf.v3/export-files! output-stream))
(catch Throwable cause
(l/err :hint "exception on exporting file"
@@ -93,10 +94,10 @@
(defn- import-binfile-v1
[{:keys [::wrk/executor] :as cfg} {:keys [project-id profile-id name file]}]
(let [cfg (-> cfg
(assoc ::bf.v1/project-id project-id)
(assoc ::bf.v1/profile-id profile-id)
(assoc ::bf.v1/name name)
(assoc ::bf.v1/input (:path file)))]
(assoc ::bfc/project-id project-id)
(assoc ::bfc/profile-id profile-id)
(assoc ::bfc/name name)
(assoc ::bfc/input (:path file)))]
;; NOTE: the importation process performs some operations that are
;; not very friendly with virtual threads, and for avoid
@@ -107,10 +108,10 @@
(defn- import-binfile-v3
[{:keys [::wrk/executor] :as cfg} {:keys [project-id profile-id name file]}]
(let [cfg (-> cfg
(assoc ::bf.v3/project-id project-id)
(assoc ::bf.v3/profile-id profile-id)
(assoc ::bf.v3/name name)
(assoc ::bf.v3/input (:path file)))]
(assoc ::bfc/project-id project-id)
(assoc ::bfc/profile-id profile-id)
(assoc ::bfc/name name)
(assoc ::bfc/input (:path file)))]
;; NOTE: the importation process performs some operations that are
;; not very friendly with virtual threads, and for avoid
;; unexpected blocking of other concurrent operations we dispatch

View File

@@ -6,6 +6,7 @@
(ns app.rpc.commands.files-update
(:require
[app.binfile.common :as bfc]
[app.common.data :as d]
[app.common.exceptions :as ex]
[app.common.features :as cfeat]
@@ -415,19 +416,37 @@
(l/error :hint "file validation error"
:cause cause))))
(defn- process-changes-and-validate
[cfg file changes skip-validate]
(let [;; WARNING: this ruins performance; maybe we need to find
;; some other way to do general validation
libs (when (and (or (contains? cf/flags :file-validation)
(contains? cf/flags :soft-file-validation))
(not skip-validate))
(get-file-libraries cfg file))
libs
(when (and (or (contains? cf/flags :file-validation)
(contains? cf/flags :soft-file-validation))
(not skip-validate))
(get-file-libraries cfg file))
file (-> (files/check-version! file)
(update :revn inc)
(update :data cpc/process-changes changes)
(update :data d/without-nils))]
;; The main purpose of this atom is provide a contextual state
;; for the changes subsystem where optionally some hints can
;; be provided for the changes processing. Right now we are
;; using it for notify about the existence of media refs when
;; a new shape is added.
state
(atom {})
file
(binding [cpc/*state* state]
(-> (files/check-version! file)
(update :revn inc)
(update :data cpc/process-changes changes)
(update :data d/without-nils)))
file
(if-let [media-refs (-> @state :media-refs not-empty)]
(bfc/update-media-references! cfg file media-refs)
file)]
(binding [pmap/*tracked* nil]
(when (contains? cf/flags :soft-file-validation)

View File

@@ -9,6 +9,7 @@
(:require
[app.binfile.common :as bfc]
[app.binfile.v1 :as bf.v1]
[app.binfile.v3 :as bf.v3]
[app.common.exceptions :as ex]
[app.common.features :as cfeat]
[app.common.schema :as sm]
@@ -25,6 +26,7 @@
[app.rpc.doc :as-alias doc]
[app.setup :as-alias setup]
[app.setup.templates :as tmpl]
[app.storage.tmp :as tmp]
[app.util.services :as sv]
[app.util.time :as dt]
[app.worker :as-alias wrk]
@@ -400,11 +402,20 @@
;; that are not very friendly with virtual threads, and for
;; avoid unexpected blocking of other concurrent operations
;; we dispatch that operation to a dedicated executor.
(let [cfg (-> cfg
(assoc ::bf.v1/project-id project-id)
(assoc ::bf.v1/profile-id profile-id)
(assoc ::bf.v1/input template))
result (px/invoke! executor (partial bf.v1/import-files! cfg))]
(let [template (tmp/tempfile-from template
:prefix "penpot.template."
:suffix ""
:min-age "30m")
format (bfc/parse-file-format template)
cfg (-> cfg
(assoc ::bfc/project-id project-id)
(assoc ::bfc/profile-id profile-id)
(assoc ::bfc/input template))
result (if (= format :binfile-v3)
(px/invoke! executor (partial bf.v3/import-files! cfg))
(px/invoke! executor (partial bf.v1/import-files! cfg)))]
(db/update! conn :project
{:modified-at (dt/now)}

View File

@@ -54,6 +54,7 @@
(::setup/templates cfg))]
(let [dest (fs/join fs/*cwd* "builtin-templates")
path (or (:path template) (fs/join dest template-id))]
(if (fs/exists? path)
(io/input-stream path)
(let [resp (http/req! cfg

View File

@@ -53,7 +53,7 @@
fixes all not propertly referenced file-media-object for a file"
[{:keys [id data] :as file} & _]
(let [conn (db/get-connection h/*system*)
used (bfc/collect-used-media data)
used (cfh/collect-used-media data)
ids (db/create-array conn "uuid" used)
sql (str "SELECT * FROM file_media_object WHERE id = ANY(?)")
rows (db/exec! conn [sql ids])

View File

@@ -16,10 +16,13 @@
[app.util.time :as dt]
[app.worker :as wrk]
[datoteka.fs :as fs]
[datoteka.io :as io]
[integrant.core :as ig]
[promesa.exec :as px]
[promesa.exec.csp :as sp])
(:import
java.io.InputStream
java.io.OutputStream
java.nio.file.Files))
(def default-tmp-dir "/tmp/penpot")
@@ -86,3 +89,12 @@
(fs/delete-on-exit! path)
(sp/offer! queue [path (some-> min-age dt/duration)])
path))
(defn tempfile-from
"Create a new tempfile from from consuming the stream"
[input & {:as options}]
(let [path (tempfile options)]
(with-open [^InputStream input (io/input-stream input)]
(with-open [^OutputStream output (io/output-stream path)]
(io/copy input output)))
path))

View File

@@ -10,7 +10,7 @@
file is eligible to be garbage collected after some period of
inactivity (the default threshold is 72h)."
(:require
[app.binfile.common :as bfc]
[app.common.files.helpers :as cfh]
[app.common.files.migrations :as fmg]
[app.common.files.validate :as cfv]
[app.common.logging :as l]
@@ -54,7 +54,7 @@
(def ^:private xf:collect-used-media
(comp
(map :data)
(mapcat bfc/collect-used-media)))
(mapcat cfh/collect-used-media)))
(defn- clean-file-media!
"Performs the garbage collection of file media objects."

View File

@@ -7,6 +7,7 @@
(ns backend-tests.binfile-test
"Internal binfile test, no RPC involved"
(:require
[app.binfile.common :as bfc]
[app.binfile.v3 :as v3]
[app.common.features :as cfeat]
[app.common.pprint :as pp]
@@ -93,15 +94,15 @@
(v3/export-files!
(-> th/*system*
(assoc ::v3/ids #{(:id file)})
(assoc ::v3/embed-assets false)
(assoc ::v3/include-libraries false))
(assoc ::bfc/ids #{(:id file)})
(assoc ::bfc/embed-assets false)
(assoc ::bfc/include-libraries false))
(io/output-stream output))
(let [result (-> th/*system*
(assoc ::v3/project-id (:default-project-id profile))
(assoc ::v3/profile-id (:id profile))
(assoc ::v3/input output)
(assoc ::bfc/project-id (:default-project-id profile))
(assoc ::bfc/profile-id (:id profile))
(assoc ::bfc/input output)
(v3/import-files!))]
(t/is (= (count result) 1))
(t/is (every? uuid? result)))))

View File

@@ -1658,3 +1658,174 @@
components (get-in result [:data :components])]
(t/is (not (contains? components c-id)))))))
(defn add-file-media-object
[& {:keys [profile-id file-id]}]
(let [mfile {:filename "sample.jpg"
:path (th/tempfile "backend_tests/test_files/sample.jpg")
:mtype "image/jpeg"
:size 312043}
params {::th/type :upload-file-media-object
::rpc/profile-id profile-id
:file-id file-id
:is-local true
:name "testfile"
:content mfile}
out (th/command! params)]
;; (th/print-result! out)
(t/is (nil? (:error out)))
(:result out)))
(t/deftest file-gc-with-media-assets-and-absorb-library
(let [storage (:app.storage/storage th/*system*)
profile (th/create-profile* 1)
file-1 (th/create-file* 1 {:profile-id (:id profile)
:project-id (:default-project-id profile)
:is-shared true})
file-2 (th/create-file* 2 {:profile-id (:id profile)
:project-id (:default-project-id profile)
:is-shared false})
fmedia (add-file-media-object :profile-id (:id profile) :file-id (:id file-1))
rel (th/link-file-to-library*
{:file-id (:id file-2)
:library-id (:id file-1)})
s-id-1 (uuid/random)
s-id-2 (uuid/random)
c-id (uuid/random)
f1-page-id (first (get-in file-1 [:data :pages]))
f2-page-id (first (get-in file-2 [:data :pages]))
fills
[{:fill-image
{:id (:id fmedia)
:name "test"
:width 200
:height 200}}]]
;; Update file library inserting new component
(update-file!
:file-id (:id file-1)
:profile-id (:id profile)
:revn 0
:vern 0
:changes
[{:type :add-obj
:page-id f1-page-id
:id s-id-1
:parent-id uuid/zero
:frame-id uuid/zero
:components-v2 true
:obj (cts/setup-shape
{:id s-id-1
:name "Board"
:frame-id uuid/zero
:parent-id uuid/zero
:type :frame
:fills fills
:main-instance true
:component-root true
:component-file (:id file-1)
:component-id c-id})}
{:type :add-component
:path ""
:name "Board"
:main-instance-id s-id-1
:main-instance-page f1-page-id
:id c-id
:anotation nil}])
;; Instanciate a component in a different file
(update-file!
:file-id (:id file-2)
:profile-id (:id profile)
:revn 0
:vern 0
:changes
[{:type :add-obj
:page-id f2-page-id
:id s-id-2
:parent-id uuid/zero
:frame-id uuid/zero
:components-v2 true
:obj (cts/setup-shape
{:id s-id-2
:name "Board"
:frame-id uuid/zero
:parent-id uuid/zero
:type :frame
:fills fills
:main-instance false
:component-root true
:component-file (:id file-1)
:component-id c-id})}])
;; Check that file media object references are present for both objects
;; the original one and the instance.
(let [rows (th/db-exec! ["SELECT * FROM file_media_object ORDER BY created_at ASC"])]
(t/is (= 2 (count rows)))
(t/is (= (:id file-1) (:file-id (get rows 0))))
(t/is (= (:id file-2) (:file-id (get rows 1))))
(t/is (every? (comp nil? :deleted-at) rows)))
;; Check if the underlying media reference on shape is different
;; from the instantiation
(let [data {::th/type :get-file
::rpc/profile-id (:id profile)
:id (:id file-2)}
out (th/command! data)]
(t/is (th/success? out))
(let [result (:result out)
fill (get-in result [:data :pages-index f2-page-id :objects s-id-2 :fills 0 :fill-image])]
(t/is (some? fill))
(t/is (not= (:id fill) (:id fmedia)))))
;; Run the file-gc on file and library
(t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-1)})))
(t/is (true? (th/run-task! :file-gc {:min-age 0 :file-id (:id file-2)})))
;; Now proceed to delete file and absorb it
(let [data {::th/type :delete-file
::rpc/profile-id (:id profile)
:id (:id file-1)}
out (th/command! data)]
(t/is (th/success? out)))
(th/run-task! :delete-object
{:object :file
:deleted-at (dt/now)
:id (:id file-1)})
;; Check that file media object references are marked all for deletion
(let [rows (th/db-exec! ["SELECT * FROM file_media_object ORDER BY created_at ASC"])]
;; (pp/pprint rows)
(t/is (= 2 (count rows)))
(t/is (= (:id file-1) (:file-id (get rows 0))))
(t/is (some? (:deleted-at (get rows 0))))
(t/is (= (:id file-2) (:file-id (get rows 1))))
(t/is (nil? (:deleted-at (get rows 1)))))
(th/run-task! :objects-gc
{:min-age 0})
(let [rows (th/db-exec! ["SELECT * FROM file_media_object ORDER BY created_at ASC"])]
(t/is (= 1 (count rows)))
(t/is (= (:id file-2) (:file-id (get rows 0))))
(t/is (nil? (:deleted-at (get rows 0)))))))

View File

@@ -484,6 +484,11 @@
modification."
nil)
(def ^:dynamic *state*
"A general purpose state to signal some out of order operations
to the processor backend."
nil)
(defmulti process-change (fn [_ change] (:type change)))
(defmulti process-operation (fn [_ op] (:type op)))
@@ -617,12 +622,38 @@
;; --- Shape / Obj
;; The main purpose of this is ensure that all created shapes has
;; valid media references; so for make sure of it, we analyze each
;; shape added via `:add-obj` change for media usage, and if shape has
;; media refs, we put that media refs on the check list (on the
;; *state*) which will subsequently be processed and all incorrect
;; references will be corrected. The media ref is anything that can
;; be pointing to a file-media-object on the shape, per example we
;; have fill-image, stroke-image, etc.
(defn- collect-shape-media-refs
[state obj page-id]
(let [media-refs
(-> (cfh/collect-shape-media-refs obj)
(not-empty))
xform
(map (fn [id]
{:page-id page-id
:shape-id (:id obj)
:id id}))]
(update state :media-refs into xform media-refs)))
(defmethod process-change :add-obj
[data {:keys [id obj page-id component-id frame-id parent-id index ignore-touched]}]
(let [update-container
(fn [container]
(ctst/add-shape id obj container frame-id parent-id index ignore-touched))]
(when *state*
(swap! *state* collect-shape-media-refs obj page-id))
(if page-id
(d/update-in-when data [:pages-index page-id] update-container)
(d/update-in-when data [:components component-id] update-container))))
@@ -876,7 +907,7 @@
(letfn [(update-fn [data]
(if (some? value)
(assoc-in data [:plugin-data namespace key] value)
(update-in data [:plugin-data namespace] dissoc key)))]
(d/update-in-when data [:plugin-data namespace] dissoc key)))]
(case object-type
:file

View File

@@ -6,4 +6,4 @@
(ns app.common.files.defaults)
(def version 59)
(def version 65)

View File

@@ -12,6 +12,7 @@
[app.common.schema :as sm]
[app.common.uuid :as uuid]
[clojure.set :as set]
[clojure.walk :as walk]
[cuerdas.core :as str]))
#?(:clj (set! *warn-on-reflection* true))
@@ -533,6 +534,86 @@
(get-position-on-parent objects)
inc))
(defn collect-shape-media-refs
"Collect all media refs on the provided shape. Returns a set of ids"
[shape]
(sequence
(keep :id)
;; NOTE: because of some bug, we ended with
;; many shape types having the ability to
;; have fill-image attribute (which initially
;; designed for :path shapes).
(concat [(:fill-image shape)
(:metadata shape)]
(map :fill-image (:fills shape))
(map :stroke-image (:strokes shape))
(->> (:content shape)
(tree-seq map? :children)
(mapcat :fills)
(map :fill-image)))))
(def ^:private
xform:collect-media-refs
"A transducer for collect media-id usage across a container (page or
component)"
(comp
(map :objects)
(mapcat vals)
(mapcat collect-shape-media-refs)))
(defn collect-used-media
"Given a fdata (file data), returns all media references used in the
file data"
[data]
(-> #{}
(into xform:collect-media-refs (vals (:pages-index data)))
(into xform:collect-media-refs (vals (:components data)))
(into (keys (:media data)))))
(defn relink-media-refs
"A function responsible to analyze all file data and replace the
old :component-file reference with the new ones, using the provided
file-index."
[data lookup-index]
(letfn [(process-map-form [form]
(cond-> form
;; Relink image shapes
(and (map? (:metadata form))
(= :image (:type form)))
(update-in [:metadata :id] lookup-index)
;; Relink paths with fill image
(map? (:fill-image form))
(update-in [:fill-image :id] lookup-index)
;; This covers old shapes and the new :fills.
(uuid? (:fill-color-ref-file form))
(update :fill-color-ref-file lookup-index)
;; This covers the old shapes and the new :strokes
(uuid? (:stroke-color-ref-file form))
(update :stroke-color-ref-file lookup-index)
;; This covers all text shapes that have typography referenced
(uuid? (:typography-ref-file form))
(update :typography-ref-file lookup-index)
;; This covers the component instance links
(uuid? (:component-file form))
(update :component-file lookup-index)
;; This covers the shadows and grids (they have directly
;; the :file-id prop)
(uuid? (:file-id form))
(update :file-id lookup-index)))
(process-form [form]
(if (map? form)
(process-map-form form)
form))]
(walk/postwalk process-form data)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; SHAPES ORGANIZATION (PATH MANAGEMENT)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

View File

@@ -25,6 +25,7 @@
[app.common.text :as txt]
[app.common.types.color :as ctc]
[app.common.types.component :as ctk]
[app.common.types.container :as ctn]
[app.common.types.file :as ctf]
[app.common.types.shape :as cts]
[app.common.types.shape.shadow :as ctss]
@@ -1145,6 +1146,57 @@
(update :pages-index update-vals update-container)
(update :components update-vals update-container))))
(defn migrate-up-62
[data]
(let [xform-cycles-ids
(comp (filter #(= (:id %) (:shape-ref %)))
(map :id))
remove-cycles
(fn [objects]
(let [cycles-ids (into #{} xform-cycles-ids (vals objects))
to-detach (->> cycles-ids
(map #(get objects %))
(map #(ctn/get-head-shape objects %))
(map :id)
distinct
(mapcat #(ctn/get-children-in-instance objects %))
(map :id)
set)]
(reduce-kv (fn [objects id shape]
(if (contains? to-detach id)
(assoc objects id (ctk/detach-shape shape))
objects))
objects
objects)))
update-component
(fn [component]
;; we only have encounter this on deleted components,
;; so the relevant objects are inside the component
(d/update-when component :objects remove-cycles))]
(update data :components update-vals update-component)))
(defn migrate-up-65
[data]
(let [update-object
(fn [object]
(d/update-when object :plugin-data d/without-nils))
update-page
(fn [page]
(-> (update-object page)
(update :objects update-vals update-object)))]
(-> data
(update-object)
(d/update-when :pages-index update-vals update-page)
(d/update-when :colors update-vals update-object)
(d/update-when :typographies update-vals update-object)
(d/update-when :components update-vals update-object))))
(def migrations
"A vector of all applicable migrations"
[{:id 2 :migrate-up migrate-up-2}
@@ -1194,5 +1246,6 @@
{:id 55 :migrate-up migrate-up-55}
{:id 56 :migrate-up migrate-up-56}
{:id 57 :migrate-up migrate-up-57}
{:id 59 :migrate-up migrate-up-59}])
{:id 59 :migrate-up migrate-up-59}
{:id 62 :migrate-up migrate-up-62}
{:id 65 :migrate-up migrate-up-65}])

View File

@@ -29,6 +29,15 @@ x-flags: &penpot-flags
x-uri: &penpot-public-uri
PENPOT_PUBLIC_URI: http://localhost:9001
x-body-size: &penpot-http-body-size
# Max body size (30MiB); Used for plain requests, should never be
# greater than multi-part size
PENPOT_HTTP_SERVER_MAX_BODY_SIZE: 31457280
# Max multipart body size (350MiB)
PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE: 367001600
networks:
penpot:
@@ -103,7 +112,7 @@ services:
# - "traefik.http.routers.penpot-https.tls.certresolver=letsencrypt"
environment:
<< : *penpot-flags
<< : [*penpot-flags, *penpot-http-body-size]
penpot-backend:
image: "penpotapp/backend:latest"
@@ -123,7 +132,7 @@ services:
## container.
environment:
<< : [*penpot-flags, *penpot-public-uri]
<< : [*penpot-flags, *penpot-public-uri, *penpot-http-body-size]
## Penpot SECRET KEY. It serves as a master key from which other keys for subsystems
## (eg http sessions, or invitations) are derived.
@@ -261,5 +270,3 @@ services:
# ports:
# - 9000:9000
# - 9001:9001

View File

@@ -22,7 +22,9 @@ update_flags /var/www/app/js/config.js
export PENPOT_BACKEND_URI=${PENPOT_BACKEND_URI:-http://penpot-backend:6060};
export PENPOT_EXPORTER_URI=${PENPOT_EXPORTER_URI:-http://penpot-exporter:6061};
export PENPOT_INTERNAL_RESOLVER=${PENPOT_INTERNAL_RESOLVER:-127.0.0.11};
export PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE=${PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE:-367001600}; # Default to 350MiB
envsubst "\$PENPOT_BACKEND_URI,\$PENPOT_EXPORTER_URI,\$PENPOT_INTERNAL_RESOLVER" < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
envsubst "\$PENPOT_BACKEND_URI,\$PENPOT_EXPORTER_URI,\$PENPOT_INTERNAL_RESOLVER,\$PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE" \
< /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
exec "$@";

View File

@@ -64,7 +64,7 @@ http {
listen 8080 default_server;
server_name _;
client_max_body_size 100M;
client_max_body_size $PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE;
charset utf-8;
proxy_http_version 1.1;

View File

@@ -7,6 +7,14 @@ title: 2. Create a Plugin
This guide covers the creation of a Penpot plugin. Penpot offers two ways to kickstart your development:
<p class="advice">
Have you got an idea for a new plugin? Great! But first take a look at <a
href="https://penpot.app/penpothub/plugins">the plugin overview</a> to see if already
exists, and consider joining efforts with other developers. This does not imply that we
won't accept plugins that do similar things, since anything can be improved and done in
different ways.
</p>
1. Using a Template:
- **Typescript template**: Using the <a target="_blank" href="https://github.com/penpot/penpot-plugin-starter-template">Penpot Plugin Starter Template</a>: A basic template with the required files for quickstarting your plugin. This template uses Typescript and Vite.

View File

@@ -405,6 +405,14 @@ where users will access the application:
PENPOT_PUBLIC_URI: http://localhost:9001
```
<p class="advice">
If you plan to serve Penpot under different domain than `localhost` without HTTPS,
you need to disable the `secure` flag on cookies, with the `disable-secure-session-cookies` flag.
This is a configuration NOT recommended for production environments.
</p>
Check all the [flags](#other-flags) to fully customize your instance.
## Frontend ##
In comparison with backend, frontend only has a small number of runtime configuration
@@ -424,8 +432,8 @@ To connect the frontend to the exporter and backend, you need to fill out these
```bash
# Frontend
PENPOT_BACKEND_URI: http://your-penpot-backend
PENPOT_EXPORTER_URI: http://your-penpot-exporter
PENPOT_BACKEND_URI: http://your-penpot-backend:6060
PENPOT_EXPORTER_URI: http://your-penpot-exporter:6061
```
These variables are used for generate correct nginx.conf file on container startup.
@@ -480,3 +488,4 @@ __Since version 2.0.0__
[2]: /technical-guide/getting-started#configure-penpot-with-docker
[3]: /technical-guide/developer/common#dev-environment
[4]: https://github.com/penpot/penpot/blob/main/docker/images/files/nginx.conf

View File

@@ -227,6 +227,9 @@ docker compose -f docker-compose.yaml pull
This will fetch the latest images. When you do <code class="language-bash">docker compose up</code> again, the containers will be recreated with the latest version.
<p class="advice">
It is strongly recommended to update the Penpot version in small increments, rather than updating between two distant versions.
</p>
**Important: Upgrade from version 1.x to 2.0**

View File

@@ -22,3 +22,22 @@ test("Bug 7549 - User clicks on color swatch to display the color picker next to
const distance = swatchBox.x - (pickerBox.x + pickerBox.width);
expect(distance).toBeLessThan(60);
});
// Fix for https://tree.taiga.io/project/penpot/issue/9900
test("Bug 9900 - Color picker has no inputs for HSV values", async ({
page,
}) => {
const workspacePage = new WorkspacePage(page);
await workspacePage.setupEmptyFile(page);
await workspacePage.goToWorkspace();
const swatch = workspacePage.page.getByRole("button", { name: "E8E9EA" });
await swatch.click();
const HSVA = await workspacePage.page.getByLabel("HSVA");
await HSVA.click();
await workspacePage.page.getByLabel("H", { exact: true }).isVisible();
await workspacePage.page.getByLabel("S", { exact: true }).isVisible();
await workspacePage.page.getByLabel("V", { exact: true }).isVisible();
});

View File

@@ -225,3 +225,16 @@ test("Bug 9066 - Problem with grid layout", async ({ page }) => {
page.getByTestId("children-6ad3e6b9-c5a0-80cf-8005-283bbe378bcb"),
).toHaveText(["CBCDEF"]);
});
test("[Taiga #9929] Paste text in workspace", async ({ page, context }) => {
const workspacePage = new WorkspacePage(page);
await workspacePage.setupEmptyFile(page);
await workspacePage.goToWorkspace();
await context.grantPermissions(["clipboard-read", "clipboard-write"]);
await page.evaluate(() => navigator.clipboard.writeText("Lorem ipsum dolor"));
await workspacePage.viewport.click({ button: "right" });
await page.getByText("PasteCtrlV").click();
await workspacePage.viewport
.getByRole("textbox")
.getByText("Lorem ipsum dolor");
});

View File

@@ -953,6 +953,7 @@
(defn create-file
[{:keys [project-id name] :as params}]
(dm/assert! (uuid? project-id))
(ptk/reify ::create-file
ev/Event
(-data [_] {:project-id project-id})

View File

@@ -1722,10 +1722,10 @@
(coll? transit-data)
(rx/of (paste-transit (assoc transit-data :in-viewport in-viewport?)))
(string? html-data)
(and (string? html-data) (d/not-empty? html-data))
(rx/of (paste-html-text html-data text-data))
(string? text-data)
(and (string? text-data) (d/not-empty? text-data))
(rx/of (paste-text text-data))
:else

View File

@@ -160,7 +160,7 @@
(let [mdata {:on-success on-file-created}
params {:project-id (:id project)}]
(st/emit! (-> (dd/create-file (with-meta params mdata))
(with-meta {::ev/origin origin}))))))]
(with-meta {::ev/origin origin :has-files (> file-count 0)}))))))]
(mf/with-effect [project]
(when project

View File

@@ -20,7 +20,10 @@
(mf/use-fn
(mf/deps create-fn)
(fn [_]
(create-fn "dashboard:empty-folder-placeholder")))]
(create-fn "dashboard:empty-folder-placeholder")))
show-text (mf/use-state nil)
on-mouse-enter (mf/use-fn #(reset! show-text true))
on-mouse-leave (mf/use-fn #(reset! show-text nil))]
(cond
(true? dragging?)
[:ul
@@ -43,9 +46,15 @@
:else
[:div {:class (stl/css :grid-empty-placeholder)}
[:button {:class (stl/css :create-new)
:on-click on-click}
(if (cf/external-feature-flag "add-file-01" "test") (tr "dashboard.add-file") i/add)]])))
(if (cf/external-feature-flag "add-file-01" "test")
[:button {:class (stl/css :create-new)
:on-click on-click
:on-mouse-enter on-mouse-enter
:on-mouse-leave on-mouse-leave}
(if @show-text (tr "dashboard.add-file") i/add)]
[:button {:class (stl/css :create-new)
:on-click on-click}
i/add])])))
(mf/defc loading-placeholder
[]

View File

@@ -179,7 +179,7 @@
(let [mdata {:on-success on-file-created}
params {:project-id project-id}]
(st/emit! (-> (dd/create-file (with-meta params mdata))
(with-meta {::ev/origin origin}))))))
(with-meta {::ev/origin origin :has-files (> file-count 0)}))))))
on-create-click
(mf/use-fn

View File

@@ -97,6 +97,8 @@
active-color-tab (mf/use-state (dc/get-active-color-tab))
drag? (mf/use-state false)
type (if (= @active-color-tab "hsva") :hsv :rgb)
fill-image-ref (mf/use-ref nil)
selected-mode (get state :type :color)
@@ -358,7 +360,7 @@
:on-change-tab on-change-tab}]]
[:& color-inputs
{:type (if (= @active-color-tab :hsva) :hsv :rgb)
{:type type
:disable-opacity disable-opacity
:color current-color
:on-change handle-change-color}]

View File

@@ -120,7 +120,7 @@
(if (= type :rgb)
[:*
[:div {:class (stl/css :input-wrapper)}
[:span {:class (stl/css :input-label)} "R"]
[:label {:for "red-value" :class (stl/css :input-label)} "R"]
[:input {:id "red-value"
:ref (:r refs)
:type "number"
@@ -129,7 +129,7 @@
:default-value red
:on-change (on-change-property :r 255)}]]
[:div {:class (stl/css :input-wrapper)}
[:span {:class (stl/css :input-label)} "G"]
[:label {:for "green-value" :class (stl/css :input-label)} "G"]
[:input {:id "green-value"
:ref (:g refs)
:type "number"
@@ -138,7 +138,7 @@
:default-value green
:on-change (on-change-property :g 255)}]]
[:div {:class (stl/css :input-wrapper)}
[:span {:class (stl/css :input-label)} "B"]
[:label {:for "blue-value" :class (stl/css :input-label)} "B"]
[:input {:id "blue-value"
:ref (:b refs)
:type "number"
@@ -149,7 +149,7 @@
[:*
[:div {:class (stl/css :input-wrapper)}
[:span {:class (stl/css :input-label)} "H"]
[:label {:for "hue-value" :class (stl/css :input-label)} "H"]
[:input {:id "hue-value"
:ref (:h refs)
:type "number"
@@ -158,7 +158,7 @@
:default-value hue
:on-change (on-change-property :h 360)}]]
[:div {:class (stl/css :input-wrapper)}
[:span {:class (stl/css :input-label)} "S"]
[:label {:for "saturation-value" :class (stl/css :input-label)} "S"]
[:input {:id "saturation-value"
:ref (:s refs)
:type "number"
@@ -168,7 +168,7 @@
:default-value saturation
:on-change (on-change-property :s 100)}]]
[:div {:class (stl/css :input-wrapper)}
[:span {:class (stl/css :input-label)} "V"]
[:label {:for "value-value" :class (stl/css :input-label)} "V"]
[:input {:id "value-value"
:ref (:v refs)
:type "number"
@@ -179,7 +179,7 @@
[:div {:class (stl/css :hex-alpha-wrapper)}
[:div {:class (stl/css-case :input-wrapper true
:hex true)}
[:span {:class (stl/css :input-label)} "HEX"]
[:label {:for "hex-value" :class (stl/css :input-label)} "HEX"]
[:input {:id "hex-value"
:ref (:hex refs)
:default-value hex
@@ -187,7 +187,7 @@
:on-blur on-blur-hex}]]
(when (not disable-opacity)
[:div {:class (stl/css-case :input-wrapper true)}
[:span {:class (stl/css :input-label)} "A"]
[:label {:for "alpha-value" :class (stl/css :input-label)} "A"]
[:input {:id "alpha-value"
:ref (:alpha refs)
:type "number"

View File

@@ -7,7 +7,6 @@
(ns app.main.ui.workspace.right-header
(:require-macros [app.main.style :as stl])
(:require
[app.config :as cf]
[app.main.data.events :as ev]
[app.main.data.modal :as modal]
[app.main.data.shortcuts :as scd]
@@ -250,9 +249,7 @@
:on-click toggle-history}
i/history]])
(when (and
(not (:is-default team))
(cf/external-feature-flag "share-01" "test"))
(when (not (:is-default team))
[:a {:class (stl/css :viewer-btn)
:title (tr "workspace.header.share")
:on-click open-share-dialog}

View File

@@ -13,9 +13,7 @@
[app.common.types.shape.impl :as shape.impl]
[app.common.types.shape.layout :as ctl]
[app.common.types.shape.radius :as ctsr]
[app.common.types.tokens-lib :as ctob]
[app.main.constants :refer [size-presets]]
[app.main.data.tokens :as dt]
[app.main.data.workspace :as udw]
[app.main.data.workspace.interactions :as dwi]
[app.main.data.workspace.shapes :as dwsh]
@@ -25,13 +23,8 @@
[app.main.ui.components.dropdown :refer [dropdown]]
[app.main.ui.components.numeric-input :refer [numeric-input*]]
[app.main.ui.components.radio-buttons :refer [radio-button radio-buttons]]
[app.main.ui.context :as muc]
[app.main.ui.hooks :as hooks]
[app.main.ui.icons :as i]
[app.main.ui.workspace.tokens.core :as wtc]
[app.main.ui.workspace.tokens.editable-select :refer [editable-select]]
[app.main.ui.workspace.tokens.style-dictionary :as sd]
[app.main.ui.workspace.tokens.token-types :as wtty]
[app.util.dom :as dom]
[app.util.i18n :as i18n :refer [tr]]
[clojure.set :refer [rename-keys union]]
@@ -92,8 +85,6 @@
(reduce #(union %1 %2) (map #(get type->options %) all-types))
(get type->options type))
design-tokens? (mf/use-ctx muc/design-tokens)
ids-with-children (or ids-with-children ids)
old-shapes (if (= type :multiple)
@@ -106,34 +97,6 @@
selection-parents-ref (mf/use-memo (mf/deps ids) #(refs/parents-by-ids ids))
selection-parents (mf/deref selection-parents-ref)
tokens (sd/use-active-theme-sets-tokens)
tokens-by-type (mf/use-memo
(mf/deps tokens)
#(ctob/group-by-type tokens))
border-radius-tokens (:border-radius tokens-by-type)
border-radius-options (mf/use-memo
(mf/deps shape border-radius-tokens)
#(wtc/tokens->select-options
{:shape shape
:tokens border-radius-tokens
:attributes (wtty/token-attributes :border-radius)}))
sizing-tokens (:sizing tokens-by-type)
width-options (mf/use-memo
(mf/deps shape sizing-tokens)
#(wtc/tokens->select-options
{:shape shape
:tokens sizing-tokens
:attributes (wtty/token-attributes :sizing)
:selected-attributes #{:width}}))
height-options (mf/use-memo
(mf/deps shape sizing-tokens)
#(wtc/tokens->select-options
{:shape shape
:tokens sizing-tokens
:attributes (wtty/token-attributes :sizing)
:selected-attributes #{:height}}))
flex-child? (->> selection-parents (some ctl/flex-layout?))
absolute? (ctl/item-absolute? shape)
flex-container? (ctl/flex-layout? shape)
@@ -247,22 +210,9 @@
(mf/use-fn
(mf/deps ids)
(fn [value attr]
(let [token-value (wtc/maybe-resolve-token-value value)
undo-id (js/Symbol)]
(binding [shape.impl/*wasm-sync* true]
(if-not design-tokens?
(st/emit! (udw/trigger-bounding-box-cloaking ids)
(udw/update-dimensions ids attr (or token-value value)))
(st/emit! (udw/trigger-bounding-box-cloaking ids)
(dwu/start-undo-transaction undo-id)
(dwsh/update-shapes ids
(if token-value
#(assoc-in % [:applied-tokens attr] (:id value))
#(d/dissoc-in % [:applied-tokens attr]))
{:reg-objects? true
:attrs [:applied-tokens]})
(udw/update-dimensions ids attr (or token-value value))
(dwu/commit-undo-transaction undo-id)))))))
(binding [shape.impl/*wasm-sync* true]
(st/emit! (udw/trigger-bounding-box-cloaking ids)
(udw/update-dimensions ids attr value)))))
on-proportion-lock-change
(mf/use-fn
@@ -335,27 +285,13 @@
(on-switch-to-radius-4)
(on-switch-to-radius-1))))
on-border-radius-token-unapply
(mf/use-fn
(mf/deps ids change-radius)
(fn [token]
(let [token-value (wtc/maybe-resolve-token-value token)]
(st/emit!
(change-radius (fn [shape]
(-> (dt/unapply-token-id shape (wtty/token-attributes :border-radius))
(ctsr/set-radius-1 token-value))))))))
on-radius-1-change
(mf/use-fn
(mf/deps ids change-radius)
(fn [value]
(let [token-value (wtc/maybe-resolve-token-value value)]
(st/emit!
(change-radius (fn [shape]
(-> (dt/maybe-apply-token-to-shape {:token (when token-value value)
:shape shape
:attributes (wtty/token-attributes :border-radius)})
(ctsr/set-radius-1 (or token-value value)))))))))
(st/emit!
(change-radius (fn [shape]
(ctsr/set-radius-1 shape value))))))
on-radius-multi-change
(mf/use-fn
@@ -464,50 +400,24 @@
:disabled disabled-width-sizing?)
:title (tr "workspace.options.width")}
[:span {:class (stl/css :icon-text)} "W"]
(if-not design-tokens?
[:> numeric-input* {:min 0.01
:no-validate true
:placeholder (if (= :multiple (:width values)) (tr "settings.multiple") "--")
:on-change on-width-change
:disabled disabled-width-sizing?
:class (stl/css :numeric-input)
:value (:width values)}]
[:& editable-select
{:placeholder (if (= :multiple (:rx values)) (tr "settings.multiple") "--")
:class (stl/css :token-select)
:disabled disabled-width-sizing?
:on-change on-width-change
:on-token-remove #(on-width-change (wtc/maybe-resolve-token-value %))
:options width-options
:position :left
:value (:width values)
:input-props {:type "number"
:no-validate true
:min 0.01}}])]
[:> numeric-input* {:min 0.01
:no-validate true
:placeholder (if (= :multiple (:width values)) (tr "settings.multiple") "--")
:on-change on-width-change
:disabled disabled-width-sizing?
:class (stl/css :numeric-input)
:value (:width values)}]]
[:div {:class (stl/css-case :height true
:disabled disabled-height-sizing?)
:title (tr "workspace.options.height")}
[:span {:class (stl/css :icon-text)} "H"]
(if-not design-tokens?
[:> numeric-input* {:min 0.01
:no-validate true
:placeholder (if (= :multiple (:height values)) (tr "settings.multiple") "--")
:on-change on-height-change
:disabled disabled-height-sizing?
:class (stl/css :numeric-input)
:value (:height values)}]
[:& editable-select
{:placeholder (if (= :multiple (:rx values)) (tr "settings.multiple") "--")
:class (stl/css :token-select)
:disabled disabled-height-sizing?
:on-change on-height-change
:on-token-remove #(on-height-change (wtc/maybe-resolve-token-value %))
:options height-options
:position :right
:value (:height values)
:input-props {:type "number"
:no-validate true
:min 0.01}}])]
[:> numeric-input* {:min 0.01
:no-validate true
:placeholder (if (= :multiple (:height values)) (tr "settings.multiple") "--")
:on-change on-height-change
:disabled disabled-height-sizing?
:class (stl/css :numeric-input)
:value (:height values)}]]
[:button {:class (stl/css-case
:lock-size-btn true
:selected (true? proportion-lock)
@@ -564,24 +474,13 @@
[:div {:class (stl/css :radius-1)
:title (tr "workspace.options.radius")}
[:span {:class (stl/css :icon)} i/corner-radius]
(if-not design-tokens?
[:> numeric-input*
{:placeholder (if (= :multiple (:rx values)) (tr "settings.multiple") "--")
:ref radius-input-ref
:min 0
:on-change on-radius-1-change
:class (stl/css :numeric-input)
:value (:rx values)}]
[:& editable-select
{:placeholder (if (= :multiple (:rx values)) (tr "settings.multiple") "--")
:class (stl/css :token-select)
:on-change on-radius-1-change
:on-token-remove on-border-radius-token-unapply
:options border-radius-options
:position :right
:value (:rx values)
:input-props {:type "number"
:min 0}}])]
[:> numeric-input*
{:placeholder (if (= :multiple (:rx values)) (tr "settings.multiple") "--")
:ref radius-input-ref
:min 0
:on-change on-radius-1-change
:class (stl/css :numeric-input)
:value (:rx values)}]]
@radius-multi?
[:div {:class (stl/css :radius-1)

View File

@@ -358,6 +358,16 @@ export class SelectionController extends EventTarget {
detail: this.#currentStyle,
})
);
} else {
const firstInline = this.#textEditor.root?.firstElementChild?.firstElementChild;
if (firstInline) {
this.#updateCurrentStyle(firstInline);
this.dispatchEvent(
new CustomEvent("stylechange", {
detail: this.#currentStyle,
}),
);
}
}
}