Compare commits

...

40 Commits

Author SHA1 Message Date
Alejandro Alonso
222481fa0d 🎉 Basic graph wasm support 2025-12-22 06:51:56 +01:00
Andrey Antukh
33c786498d Merge remote-tracking branch 'origin/staging-render' into develop 2025-12-12 12:19:49 +01:00
Andrey Antukh
1f886b1f88 Merge remote-tracking branch 'origin/staging' into develop 2025-12-12 12:16:41 +01:00
Aitor Moreno
5a922c6bd6 Merge pull request #7960 from penpot/superalex-fix-too-many-active-webgl-contexts
🐛 Fix too many active WEBGL contexts
2025-12-12 12:03:46 +01:00
Alejandro Alonso
1388865cfc 🐛 Fix too many active WEBGL contexts 2025-12-12 11:16:47 +01:00
Andrey Antukh
1738847694 Merge remote-tracking branch 'origin/staging' into staging-render 2025-12-12 10:20:21 +01:00
Aitor Moreno
ca1c3c799d Merge pull request #7968 from penpot/alotor-fix-border-radius
🐛 Fix problem with border radius to path
2025-12-12 10:18:07 +01:00
alonso.torres
ce5006ae84 🐛 Fix problem with border radius to path 2025-12-11 22:40:44 +01:00
Eva Marco
50dbe6ab12 🐛 Fix horizontal scroll on layer panel (#7956) 2025-12-11 21:34:18 +01:00
Belén Albeza
0a7a65af5d ♻️ Make SerializableResult to depend on From traits 2025-12-11 16:00:03 +01:00
alonso.torres
ea4d0e1238 Calculate position data in wasm 2025-12-11 16:00:03 +01:00
Elena Torro
b705cf953a 🐛 Set layout data from set-object 2025-12-11 14:52:32 +01:00
Alejandro Alonso
90ce1f56e7 Merge pull request #7958 from penpot/superalex-fix-svg-extract-ids
🐛 Fix svg extract ids
2025-12-11 14:02:05 +01:00
Alejandro Alonso
ab0438cc6f 🐛 Fix svg extract ids 2025-12-11 13:47:00 +01:00
Aitor Moreno
c6aa9cc4b7 Merge pull request #7950 from penpot/ladybenko-12851-fix-text-selection
🐛 Fix text selection when editor regains focus
2025-12-11 13:45:29 +01:00
Andrey Antukh
5779adef33 Merge remote-tracking branch 'origin/staging' into staging-render 2025-12-11 13:30:59 +01:00
Andrey Antukh
2f46cbc0d4 Make render wasm import on worker http cache aware 2025-12-11 13:27:20 +01:00
Elena Torró
ebf1758958 Merge pull request #7935 from penpot/superalex-improve-svg-import
🎉 Improve svg import
2025-12-11 13:21:29 +01:00
Elena Torró
e94c56bfa7 Merge pull request #7954 from penpot/azazeln28-fix-font-weight-mixed-value
🐛 Fix font weight mixed value
2025-12-11 12:43:53 +01:00
Andrey Antukh
53be6f996b 🐛 Fix issues on build processs related to render-wasm 2025-12-11 12:41:19 +01:00
Alejandro Alonso
89d9591011 🎉 Improve svg import 2025-12-11 12:02:34 +01:00
Andrey Antukh
5a260294a1 🔧 Update build-tag.yml github workflow 2025-12-11 12:00:42 +01:00
Andrey Antukh
3becfcd723 🔧 Update build-tag.yml github workflow 2025-12-11 11:59:16 +01:00
Andrey Antukh
3f6e44316e 🐛 Add missing node depes install on render-wasm 2025-12-11 11:51:47 +01:00
Aitor Moreno
5501a2815f 🐛 Fix font-variant-id mixed value 2025-12-11 11:32:27 +01:00
Eva Marco
77ef8e6fe6 🐛 Fix scroll on move library modal (#7952) 2025-12-11 10:46:54 +01:00
Alejandro Alonso
1066438b02 Merge pull request #7922 from penpot/elenatorro-12855-improve-pan-rendering
🔧 Improve pan rendering
2025-12-10 15:58:59 +01:00
Alejandro Alonso
3b23a3ad19 Merge pull request #7947 from penpot/elenatorro-12880-fix-variant-ui
🔧 Support variants interactivity on the new render's UI
2025-12-10 15:27:48 +01:00
Andrey Antukh
7396f4bfb6 Merge remote-tracking branch 'origin/staging' into develop 2025-12-10 15:17:50 +01:00
Alejandro Alonso
916b7709dc Update Pencil Penpot Design System System template in carousel (#7948) 2025-12-10 15:09:28 +01:00
Belén Albeza
5cf51f3d26 🐛 Fix text selection not being restore if it was only 1 word 2025-12-10 15:05:13 +01:00
Belén Albeza
25acad5154 🔧 Add formatting rules to the TextEditor 2025-12-10 15:04:34 +01:00
Elena Torro
0a212b6291 🔧 Support variants interactivity on the new render's UI 2025-12-10 14:39:59 +01:00
Eva Marco
443e41fea4 🐛 Fix multiple selection with color tokens (#7941) 2025-12-10 14:36:08 +01:00
Alejandro Alonso
c7c9b04095 Merge pull request #7944 from penpot/niwinz-staging-exporter-fix
🐛 Fix incorrect resource lifetime handling on exporter
2025-12-10 14:35:20 +01:00
Eva Marco
c61a0c0332 📚 Add line to changelog (#7945) 2025-12-10 13:58:18 +01:00
Andrey Antukh
34e84ee3c8 🐛 Fix incorrect resource lifetime handling on exporter 2025-12-10 13:02:31 +01:00
Elena Torro
81bc1bb0af 🔧 Log performance when building using profile-macros 2025-12-09 15:25:13 +01:00
Elena Torro
b8feb6374d 🔧 Rebuild indices on zoom change, not pan 2025-12-09 11:26:03 +01:00
Elena Torro
0889df8e08 🔧 Skip slow operations on fast render 2025-12-09 11:26:03 +01:00
2694 changed files with 1261166 additions and 619 deletions

View File

@@ -11,7 +11,7 @@ jobs:
secrets: inherit
with:
gh_ref: ${{ github.ref_name }}
build_wasm: "no"
build_wasm: "yes"
build_storybook: "yes"
build-docker:

View File

@@ -114,6 +114,8 @@ example. It's still usable as before, we just removed the example.
- Fix unpublish library modal not scrolling a long file list [Taiga #12285](https://tree.taiga.io/project/penpot/issue/12285)
- Fix incorrect interaction betwen hower and scroll on assets sidebar [Taiga #12389](https://tree.taiga.io/project/penpot/issue/12389)
- Fix switch variants with paths [Taiga #12841](https://tree.taiga.io/project/penpot/issue/12841)
- Fix referencing typography tokens on font-family tokens [Taiga #12492](https://tree.taiga.io/project/penpot/issue/12492)
- Fix horizontal scroll on layer panel [Taiga #12843](https://tree.taiga.io/project/penpot/issue/12843)
## 2.11.1

View File

@@ -3,7 +3,7 @@
:file-uri "https://github.com/penpot/penpot-files/raw/refs/heads/main/Tokens%20starter%20kit.penpot"}
{:id "penpot-design-system"
:name "Penpot Design System | Pencil"
:file-uri "https://github.com/penpot/penpot-files/raw/refs/heads/main/penpot-app.penpot"}
:file-uri "https://github.com/penpot/penpot-files/raw/refs/heads/main/Pencil-Penpot-Design-System.penpot"}
{:id "wireframing-kit"
:name "Wireframe library"
:file-uri "https://github.com/penpot/penpot-files/raw/refs/heads/main/Wireframing%20kit%20v1.1.penpot"}

View File

@@ -56,6 +56,7 @@
"text-editor/v2-html-paste"
"text-editor/v2"
"render-wasm/v1"
"graph-wasm/v1"
"variants/v1"})
;; A set of features enabled by default
@@ -79,7 +80,8 @@
"text-editor/v2-html-paste"
"text-editor/v2"
"tokens/numeric-input"
"render-wasm/v1"})
"render-wasm/v1"
"graph-wasm/v1"})
;; Features that are mainly backend only or there are a proper
;; fallback when frontend reports no support for it
@@ -128,6 +130,7 @@
:feature-text-editor-v2 "text-editor/v2"
:feature-text-editor-v2-html-paste "text-editor/v2-html-paste"
:feature-render-wasm "render-wasm/v1"
:feature-graph-wasm "graph-wasm/v1"
:feature-variants "variants/v1"
:feature-token-input "tokens/numeric-input"
nil))

View File

@@ -82,6 +82,113 @@
(declare create-svg-children)
(declare parse-svg-element)
(defn- process-gradient-stops
"Processes gradient stops to extract stop-color and stop-opacity from style attributes
and convert them to direct attributes. This ensures stops with style='stop-color:#...;stop-opacity:1'
are properly converted to stop-color and stop-opacity attributes."
[stops]
(mapv (fn [stop]
(let [stop-attrs (:attrs stop)
stop-style (get stop-attrs :style)
;; Parse style if it's a string using csvg/parse-style utility
parsed-style (when (and (string? stop-style) (seq stop-style))
(csvg/parse-style stop-style))
;; Extract stop-color and stop-opacity from style
style-stop-color (when parsed-style (:stop-color parsed-style))
style-stop-opacity (when parsed-style (:stop-opacity parsed-style))
;; Merge: use direct attributes first, then style values as fallback
final-attrs (cond-> stop-attrs
(and style-stop-color (not (contains? stop-attrs :stop-color)))
(assoc :stop-color style-stop-color)
(and style-stop-opacity (not (contains? stop-attrs :stop-opacity)))
(assoc :stop-opacity style-stop-opacity)
;; Remove style attribute if we've extracted its values
(or style-stop-color style-stop-opacity)
(dissoc :style))]
(assoc stop :attrs final-attrs)))
stops))
(defn- resolve-gradient-href
"Resolves xlink:href references in gradients by merging the referenced gradient's
stops and attributes with the referencing gradient. This ensures gradients that
reference other gradients (like linearGradient3550 referencing linearGradient3536)
inherit the stops from the base gradient.
According to SVG spec, when a gradient has xlink:href:
- It inherits all attributes from the referenced gradient
- It inherits all stops from the referenced gradient
- The referencing gradient's attributes override the base ones
- If the referencing gradient has stops, they replace the base stops
Returns the defs map with all gradient href references resolved."
[defs]
(letfn [(resolve-gradient [gradient-id gradient-node defs visited]
(if (contains? visited gradient-id)
(do
#?(:cljs (js/console.warn "[resolve-gradient] Circular reference detected for" gradient-id)
:clj nil)
gradient-node) ;; Avoid circular references
(let [attrs (:attrs gradient-node)
href-id (or (:href attrs) (:xlink:href attrs))
href-id (when (and (string? href-id) (pos? (count href-id)))
(subs href-id 1)) ;; Remove leading #
base-gradient (when (and href-id (contains? defs href-id))
(get defs href-id))
resolved-base (when base-gradient (resolve-gradient href-id base-gradient defs (conj visited gradient-id)))]
(if resolved-base
;; Merge: base gradient attributes + referencing gradient attributes
;; Use referencing gradient's stops if present, otherwise use base stops
(let [base-attrs (:attrs resolved-base)
ref-attrs (:attrs gradient-node)
;; Start with base attributes (without id), then merge with ref attributes
;; This ensures ref attributes override base ones
base-attrs-clean (dissoc base-attrs :id)
ref-attrs-clean (dissoc ref-attrs :href :xlink:href :id)
;; Special handling for gradientTransform: if both have it, combine them
base-transform (get base-attrs :gradientTransform)
ref-transform (get ref-attrs :gradientTransform)
combined-transform (cond
(and base-transform ref-transform)
(str base-transform " " ref-transform) ;; Apply base first, then ref
:else (or ref-transform base-transform))
;; Merge attributes: base first, then ref (ref overrides)
merged-attrs (-> (d/deep-merge base-attrs-clean ref-attrs-clean)
(cond-> combined-transform
(assoc :gradientTransform combined-transform)))
;; If referencing gradient has content (stops), use it; otherwise use base content
final-content (if (seq (:content gradient-node))
(:content gradient-node)
(:content resolved-base))
;; Process stops to extract stop-color and stop-opacity from style attributes
processed-content (process-gradient-stops final-content)
result {:tag (:tag gradient-node)
:attrs (assoc merged-attrs :id gradient-id)
:content processed-content}]
result)
;; Process stops even for gradients without references to extract style attributes
(let [processed-content (process-gradient-stops (:content gradient-node))]
(assoc gradient-node :content processed-content))))))]
(let [gradient-tags #{:linearGradient :radialGradient}
result (reduce-kv
(fn [acc id node]
(if (contains? gradient-tags (:tag node))
(assoc acc id (resolve-gradient id node defs #{}))
(assoc acc id node)))
{}
defs)]
result)))
(defn create-svg-shapes
([svg-data pos objects frame-id parent-id selected center?]
(create-svg-shapes (uuid/next) svg-data pos objects frame-id parent-id selected center?))
@@ -112,6 +219,9 @@
(csvg/fix-percents)
(csvg/extract-defs))
;; Resolve gradient href references in all defs before processing shapes
def-nodes (resolve-gradient-href def-nodes)
;; In penpot groups have the size of their children. To
;; respect the imported svg size and empty space let's create
;; a transparent shape as background to respect the imported
@@ -142,12 +252,23 @@
(reduce (partial create-svg-children objects selected frame-id root-id svg-data)
[unames []]
(d/enumerate (->> (:content svg-data)
(mapv #(csvg/inherit-attributes root-attrs %)))))]
(mapv #(csvg/inherit-attributes root-attrs %)))))
[root-shape children])))
;; Collect all defs from children and merge into root shape
all-defs-from-children (reduce (fn [acc child]
(if-let [child-defs (:svg-defs child)]
(merge acc child-defs)
acc))
{}
children)
;; Merge defs from svg-data and children into root shape
root-shape-with-defs (assoc root-shape :svg-defs (merge def-nodes all-defs-from-children))]
[root-shape-with-defs children])))
(defn create-raw-svg
[name frame-id {:keys [x y width height offset-x offset-y]} {:keys [attrs] :as data}]
[name frame-id {:keys [x y width height offset-x offset-y defs] :as svg-data} {:keys [attrs] :as data}]
(let [props (csvg/attrs->props attrs)
vbox (grc/make-rect offset-x offset-y width height)]
(cts/setup-shape
@@ -160,10 +281,11 @@
:y y
:content data
:svg-attrs props
:svg-viewbox vbox})))
:svg-viewbox vbox
:svg-defs defs})))
(defn create-svg-root
[id frame-id parent-id {:keys [name x y width height offset-x offset-y attrs]}]
[id frame-id parent-id {:keys [name x y width height offset-x offset-y attrs defs] :as svg-data}]
(let [props (-> (dissoc attrs :viewBox :view-box :xmlns)
(d/without-keys csvg/inheritable-props)
(csvg/attrs->props))]
@@ -177,7 +299,8 @@
:height height
:x (+ x offset-x)
:y (+ y offset-y)
:svg-attrs props})))
:svg-attrs props
:svg-defs defs})))
(defn create-svg-children
[objects selected frame-id parent-id svg-data [unames children] [_index svg-element]]
@@ -198,7 +321,7 @@
(defn create-group
[name frame-id {:keys [x y width height offset-x offset-y] :as svg-data} {:keys [attrs]}]
[name frame-id {:keys [x y width height offset-x offset-y defs] :as svg-data} {:keys [attrs]}]
(let [transform (csvg/parse-transform (:transform attrs))
attrs (-> attrs
(d/without-keys csvg/inheritable-props)
@@ -214,7 +337,8 @@
:height height
:svg-transform transform
:svg-attrs attrs
:svg-viewbox vbox})))
:svg-viewbox vbox
:svg-defs defs})))
(defn create-path-shape [name frame-id svg-data {:keys [attrs] :as data}]
(when (and (contains? attrs :d) (seq (:d attrs)))
@@ -523,6 +647,21 @@
:else (dm/str tag))]
(dm/str "svg-" suffix)))
(defn- filter-valid-def-references
"Filters out false positive references that are not valid def IDs.
Filters out:
- Colors in style attributes (hex colors like #f9dd67)
- Style fragments that contain CSS keywords (like stop-opacity)
- References that don't exist in defs"
[ref-ids defs]
(let [is-style-fragment? (fn [ref-id]
(or (clr/hex-color-string? (str "#" ref-id))
(str/includes? ref-id ";") ;; Contains CSS separator
(str/includes? ref-id "stop-opacity") ;; CSS keyword
(str/includes? ref-id "stop-color")))] ;; CSS keyword
(->> ref-ids
(remove is-style-fragment?) ;; Filter style fragments and hex colors
(filter #(contains? defs %))))) ;; Only existing defs
(defn parse-svg-element
[frame-id svg-data {:keys [tag attrs hidden] :as element} unames]
@@ -534,7 +673,11 @@
(let [name (or (:id attrs) (tag->name tag))
att-refs (csvg/find-attr-references attrs)
defs (get svg-data :defs)
references (csvg/find-def-references defs att-refs)
valid-refs (filter-valid-def-references att-refs defs)
all-refs (csvg/find-def-references defs valid-refs)
;; Filter the final result to ensure all references are valid defs
;; This prevents false positives from style attributes in gradient stops
references (filter-valid-def-references all-refs defs)
href-id (or (:href attrs) (:xlink:href attrs) " ")
href-id (if (and (string? href-id)

View File

@@ -546,9 +546,19 @@
filter-values)))
(defn extract-ids [val]
(when (some? val)
;; Extract referenced ids from string values like "url(#myId)".
;; Non-string values (maps, numbers, nil, etc.) return an empty seq
;; to avoid re-seq type errors when attributes carry nested structures.
(cond
(string? val)
(->> (re-seq xml-id-regex val)
(mapv second))))
(mapv second))
(sequential? val)
(mapcat extract-ids val)
:else
[]))
(defn fix-dot-number
"Fixes decimal numbers starting in dot but without leading 0"

View File

@@ -21,6 +21,7 @@
"raw-body": "^3.0.1",
"source-map-support": "^0.5.21",
"svgo": "penpot/svgo#v3.1",
"undici": "^7.16.0",
"xml-js": "^1.6.11",
"xregexp": "^5.1.2"
},

View File

@@ -7,5 +7,4 @@ bb -i '(babashka.wait/wait-for-port "localhost" 9630)';
bb -i '(babashka.wait/wait-for-path "target/app.js")';
sleep 2;
export NODE_TLS_REJECT_UNAUTHORIZED=0
exec node target/app.js

View File

@@ -107,12 +107,12 @@
:on-progress on-progress)
append (fn [{:keys [filename path] :as resource}]
(rsc/add-to-zip! zip path (str/replace filename sanitize-file-regex "_")))
(rsc/add-to-zip zip path (str/replace filename sanitize-file-regex "_")))
proc (->> exports
(map (fn [export] (rd/render export append)))
(p/all)
(p/fnly (fn [_] (.finalize zip)))
(p/mcat (fn [_] (rsc/close-zip zip)))
(p/fmap (constantly resource))
(p/mcat (partial rsc/upload-resource auth-token))
(p/fmap (fn [resource]

View File

@@ -11,6 +11,7 @@
["node:fs" :as fs]
["node:fs/promises" :as fsp]
["node:path" :as path]
["undici" :as http]
[app.common.exceptions :as ex]
[app.common.transit :as t]
[app.common.uri :as u]
@@ -53,30 +54,40 @@
(.pipe zip out)
zip))
(defn add-to-zip!
(defn add-to-zip
[zip path name]
(.file ^js zip path #js {:name name}))
(defn close-zip!
(defn close-zip
[zip]
(.finalize ^js zip))
(p/create (fn [resolve]
(.on ^js zip "close" resolve)
(.finalize ^js zip))))
(defn upload-resource
[auth-token resource]
(->> (fsp/readFile (:path resource))
(p/fmap (fn [buffer]
(js/console.log buffer)
(new js/Blob #js [buffer] #js {:type (:mtype resource)})))
(p/mcat (fn [blob]
(let [fdata (new js/FormData)
uri (-> (cf/get :public-uri)
(u/ensure-path-slash)
(u/join "api/management/methods/upload-tempfile")
(str))]
(let [fdata (new http/FormData)
agent (new http/Agent #js {:connect #js {:rejectUnauthorized false}})
headers #js {"X-Shared-Key" cf/management-key
"Authorization" (str "Bearer " auth-token)}
request #js {:headers headers
:method "POST"
:body fdata
:dispatcher agent}
uri (-> (cf/get :public-uri)
(u/ensure-path-slash)
(u/join "api/management/methods/upload-tempfile")
(str))]
(.append fdata "content" blob (:filename resource))
(js/fetch uri #js {:headers #js {"X-Shared-Key" cf/management-key
"Authorization" (str "Bearer " auth-token)}
:method "POST"
:body fdata}))))
(http/fetch uri request))))
(p/mcat (fn [response]
(if (not= (.-status response) 200)
(ex/raise :type :internal

View File

@@ -75,7 +75,8 @@
[path]
(->> (.stat fs/promises path)
(p/fmap (fn [data]
{:created-at (inst-ms (.-ctime ^js data))
{:path path
:created-at (inst-ms (.-ctime ^js data))
:size (.-size data)}))
(p/merr (fn [_cause]
(p/resolved nil)))))

View File

@@ -582,6 +582,7 @@ __metadata:
raw-body: "npm:^3.0.1"
source-map-support: "npm:^0.5.21"
svgo: "penpot/svgo#v3.1"
undici: "npm:^7.16.0"
ws: "npm:^8.18.3"
xml-js: "npm:^1.6.11"
xregexp: "npm:^5.1.2"
@@ -1513,6 +1514,13 @@ __metadata:
languageName: node
linkType: hard
"undici@npm:^7.16.0":
version: 7.16.0
resolution: "undici@npm:7.16.0"
checksum: 10c0/efd867792e9f233facf9efa0a087e2d9c3e4415c0b234061b9b40307ca4fa01d945fee4d43c7b564e1b80e0d519bcc682f9f6e0de13c717146c00a80e2f1fb0f
languageName: node
linkType: hard
"unique-filename@npm:^4.0.0":
version: 4.0.0
resolution: "unique-filename@npm:4.0.0"

View File

@@ -32,8 +32,8 @@
"e2e:server": "node ./scripts/e2e-server.js",
"fmt:clj": "cljfmt fix --parallel=true src/ test/",
"fmt:clj:check": "cljfmt check --parallel=false src/ test/",
"fmt:js": "yarn run prettier -c src/**/*.stories.jsx -c playwright/**/*.js -c scripts/**/*.js -w",
"fmt:js:check": "yarn run prettier -c src/**/*.stories.jsx -c playwright/**/*.js -c scripts/**/*.js",
"fmt:js": "yarn run prettier -c src/**/*.stories.jsx -c playwright/**/*.js -c scripts/**/*.js -c text-editor/**/*.js -w",
"fmt:js:check": "yarn run prettier -c src/**/*.stories.jsx -c playwright/**/*.js -c scripts/**/*.js text-editor/**/*.js",
"lint:clj": "clj-kondo --parallel --lint src/",
"lint:scss": "yarn run prettier -c resources/styles -c src/**/*.scss",
"lint:scss:fix": "yarn run prettier -c resources/styles -c src/**/*.scss -w",

View File

@@ -1,12 +1,12 @@
export class Clipboard {
static Permission = {
ONLY_READ: ['clipboard-read'],
ONLY_WRITE: ['clipboard-write'],
ALL: ['clipboard-read', 'clipboard-write']
}
ONLY_READ: ["clipboard-read"],
ONLY_WRITE: ["clipboard-write"],
ALL: ["clipboard-read", "clipboard-write"],
};
static enable(context, permissions) {
return context.grantPermissions(permissions)
return context.grantPermissions(permissions);
}
static writeText(page, text) {
@@ -18,8 +18,8 @@ export class Clipboard {
}
constructor(page, context) {
this.page = page
this.context = context
this.page = page;
this.context = context;
}
enable(permissions) {

View File

@@ -1,18 +1,16 @@
export class Transit {
static parse(value) {
if (typeof value !== 'string')
return value
if (typeof value !== "string") return value;
if (value.startsWith('~'))
return value.slice(2)
if (value.startsWith("~")) return value.slice(2);
return value
return value;
}
static get(object, ...path) {
let aux = object;
for (const name of path) {
if (typeof name !== 'string') {
if (typeof name !== "string") {
if (!(name in aux)) {
return undefined;
}

View File

@@ -9,7 +9,7 @@ export class BasePage {
*/
static async mockRPCs(page, paths, options) {
for (const [path, jsonFilename] of Object.entries(paths)) {
await this.mockRPC(page, path, jsonFilename, options)
await this.mockRPC(page, path, jsonFilename, options);
}
}

View File

@@ -1,7 +1,7 @@
import { expect } from "@playwright/test";
import { readFile } from 'node:fs/promises';
import { readFile } from "node:fs/promises";
import { BaseWebSocketPage } from "./BaseWebSocketPage";
import { Transit } from '../../helpers/Transit';
import { Transit } from "../../helpers/Transit";
export class WorkspacePage extends BaseWebSocketPage {
static TextEditor = class TextEditor {

View File

@@ -51,7 +51,7 @@ test.skip("BUG 12164 - Crash when trying to fetch a missing font", async ({
pageId: "2b7f0188-51a1-8193-8006-e05bad87b74d",
});
await workspacePage.page.waitForTimeout(1000)
await workspacePage.page.waitForTimeout(1000);
await workspacePage.waitForFirstRender();
await expect(

View File

@@ -1,5 +1,5 @@
import { test, expect } from "@playwright/test";
import { Clipboard } from '../../helpers/Clipboard';
import { Clipboard } from "../../helpers/Clipboard";
import { WorkspacePage } from "../pages/WorkspacePage";
const timeToWait = 100;
@@ -11,14 +11,14 @@ test.beforeEach(async ({ page, context }) => {
await WorkspacePage.mockConfigFlags(page, ["enable-feature-text-editor-v2"]);
});
test.afterEach(async ({ context}) => {
test.afterEach(async ({ context }) => {
context.clearPermissions();
})
});
test("Create a new text shape", async ({ page }) => {
const initialText = "Lorem ipsum";
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.goToWorkspace();
@@ -36,10 +36,7 @@ test("Create a new text shape from pasting text", async ({ page, context }) => {
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockRPC(
"update-file?id=*",
"text-editor/update-file.json",
);
await workspace.mockRPC("update-file?id=*", "text-editor/update-file.json");
await workspace.goToWorkspace();
await Clipboard.writeText(page, textToPaste);
@@ -55,10 +52,13 @@ test("Create a new text shape from pasting text", async ({ page, context }) => {
await workspace.textEditor.stopEditing();
});
test("Create a new text shape from pasting text using context menu", async ({ page, context }) => {
test("Create a new text shape from pasting text using context menu", async ({
page,
context,
}) => {
const textToPaste = "Lorem ipsum";
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.goToWorkspace();
@@ -72,11 +72,13 @@ test("Create a new text shape from pasting text using context menu", async ({ pa
expect(textContent).toBe(textToPaste);
await workspace.textEditor.stopEditing();
})
});
test("Update an already created text shape by appending text", async ({ page }) => {
test("Update an already created text shape by appending text", async ({
page,
}) => {
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -94,7 +96,7 @@ test("Update an already created text shape by prepending text", async ({
page,
}) => {
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -112,7 +114,7 @@ test("Update an already created text shape by inserting text in between", async
page,
}) => {
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -126,10 +128,13 @@ test("Update an already created text shape by inserting text in between", async
await workspace.textEditor.stopEditing();
});
test("Update a new text shape appending text by pasting text", async ({ page, context }) => {
test("Update a new text shape appending text by pasting text", async ({
page,
context,
}) => {
const textToPaste = " dolor sit amet";
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -147,11 +152,12 @@ test("Update a new text shape appending text by pasting text", async ({ page, co
});
test("Update a new text shape prepending text by pasting text", async ({
page, context
page,
context,
}) => {
const textToPaste = "Dolor sit amet ";
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -173,7 +179,7 @@ test("Update a new text shape replacing (starting) text with pasted text", async
}) => {
const textToPaste = "Dolor sit amet";
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -197,7 +203,7 @@ test("Update a new text shape replacing (ending) text with pasted text", async (
}) => {
const textToPaste = "dolor sit amet";
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -221,7 +227,7 @@ test("Update a new text shape replacing (in between) text with pasted text", asy
}) => {
const textToPaste = "dolor sit amet";
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
@@ -244,14 +250,11 @@ test("Update text font size selecting a part of it (starting)", async ({
page,
}) => {
const workspace = new WorkspacePage(page, {
textEditor: true
textEditor: true,
});
await workspace.setupEmptyFile();
await workspace.mockGetFile("text-editor/get-file-lorem-ipsum.json");
await workspace.mockRPC(
"update-file?id=*",
"text-editor/update-file.json",
);
await workspace.mockRPC("update-file?id=*", "text-editor/update-file.json");
await workspace.goToWorkspace();
await workspace.clickLeafLayer("Lorem ipsum");
await workspace.textEditor.startEditing();
@@ -280,7 +283,10 @@ test.skip("Update text line height selecting a part of it (starting)", async ({
await workspace.textEditor.selectFromStart(5);
await workspace.textEditor.changeLineHeight(1.4);
const lineHeight = await workspace.textEditor.waitForParagraphStyle(1, 'line-height');
const lineHeight = await workspace.textEditor.waitForParagraphStyle(
1,
"line-height",
);
expect(lineHeight).toBe("1.4");
const textContent = await workspace.textEditor.waitForTextSpanContent();

View File

@@ -0,0 +1,23 @@
<!DOCTYPE html>
<html lang="es">
<head>
<meta charset="UTF-8" />
</head>
<body>
<script type="module">
import initWasmModule from '/js/graph-wasm.js';
let Module = null;
function init(moduleInstance) {
Module = moduleInstance;
}
console.log("Loading module");
initWasmModule().then(Module => {
init(Module);
Module._hello();
});
</script>
</body>
</html>

View File

@@ -20,25 +20,26 @@ echo $PATH
set -ex
corepack enable;
corepack install || exit 1;
corepack install;
yarn install || exit 1;
rm -rf resources/public;
rm -rf target/dist;
rm -rf resources/public;
mkdir -p resources/public;
mkdir -p target/dist;
pushd ../render-wasm;
./build
popd
yarn run build:app:main $EXTRA_PARAMS;
yarn run build:app:libs;
yarn run build:app:assets;
if [ "$INCLUDE_WASM" = "yes" ]; then
yarn run build:wasm || exit 1;
fi
sed -i "s/\.\/render.js/.\/render.js?version=$CURRENT_VERSION/g" resources/public/js/worker/main*.js
yarn run build:app:libs || exit 1;
yarn run build:app:assets || exit 1;
rsync -avr resources/public/ target/dist/;
rsync -avr resources/public/ target/dist/
if [ "$INCLUDE_STORYBOOK" = "yes" ]; then
# build storybook

View File

@@ -92,7 +92,7 @@
{:main
{:entries [app.worker]
:web-worker true
:prepend-js "importScripts('/js/worker/render.js');"
:prepend-js "importScripts('./render.js', './graph-wasm-worker.js');"
:depends-on #{}}}
:js-options

View File

@@ -0,0 +1,12 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns app.graph-wasm
"A WASM based render API"
(:require
[app.graph-wasm.api :as wasm.api]))
(def module wasm.api/module)

View File

@@ -0,0 +1,91 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns app.graph-wasm.api
(:require
[app.common.data.macros :as dm]
[app.common.uuid :as uuid]
[app.config :as cf]
[app.graph-wasm.wasm :as wasm]
[app.render-wasm.helpers :as h]
[app.render-wasm.serializers :as sr]
[app.util.modules :as mod]
[promesa.core :as p]))
(defn hello []
(h/call wasm/internal-module "_hello"))
(defn init []
(h/call wasm/internal-module "_init"))
(defn use-shape
[id]
(let [buffer (uuid/get-u32 id)]
(println "use-shape" id)
(h/call wasm/internal-module "_use_shape"
(aget buffer 0)
(aget buffer 1)
(aget buffer 2)
(aget buffer 3))))
(defn set-shape-parent-id
[id]
(let [buffer (uuid/get-u32 id)]
(h/call wasm/internal-module "_set_shape_parent"
(aget buffer 0)
(aget buffer 1)
(aget buffer 2)
(aget buffer 3))))
(defn set-shape-type
[type]
(h/call wasm/internal-module "_set_shape_type" (sr/translate-shape-type type)))
(defn set-shape-selrect
[selrect]
(h/call wasm/internal-module "_set_shape_selrect"
(dm/get-prop selrect :x1)
(dm/get-prop selrect :y1)
(dm/get-prop selrect :x2)
(dm/get-prop selrect :y2)))
(defn set-object
[shape]
(let [id (dm/get-prop shape :id)
type (dm/get-prop shape :type)
parent-id (get shape :parent-id)
selrect (get shape :selrect)
children (get shape :shapes)]
(use-shape id)
(set-shape-type type)
(set-shape-parent-id parent-id)
(set-shape-selrect selrect)))
(defn set-objects
[objects]
(doseq [shape (vals objects)]
(set-object shape)))
(defn init-wasm-module
[module]
(let [default-fn (unchecked-get module "default")
href (cf/resolve-href "js/graph-wasm.wasm")]
(default-fn #js {:locateFile (constantly href)})))
(defonce module
(delay
(if (exists? js/dynamicImport)
(let [uri (cf/resolve-href "js/graph-wasm.js")]
(->> (mod/import uri)
(p/mcat init-wasm-module)
(p/fmap (fn [default]
(set! wasm/internal-module default)
true))
(p/merr
(fn [cause]
(js/console.error cause)
(p/resolved false)))))
(p/resolved false))))

View File

@@ -0,0 +1,9 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns app.graph-wasm.wasm)
(defonce internal-module #js {})

View File

@@ -24,6 +24,8 @@
(def revn-data (atom {}))
(def queue-conj (fnil conj #queue []))
(def force-persist? #(= % ::force-persist))
(defn- update-status
[status]
(ptk/reify ::update-status

View File

@@ -32,7 +32,7 @@
[app.main.data.helpers :as dsh]
[app.main.data.modal :as modal]
[app.main.data.notifications :as ntf]
[app.main.data.persistence :as-alias dps]
[app.main.data.persistence :as dps]
[app.main.data.plugins :as dp]
[app.main.data.profile :as du]
[app.main.data.project :as dpj]
@@ -67,6 +67,7 @@
[app.main.errors]
[app.main.features :as features]
[app.main.features.pointer-map :as fpmap]
[app.main.refs :as refs]
[app.main.repo :as rp]
[app.main.router :as rt]
[app.render-wasm :as wasm]
@@ -379,6 +380,59 @@
(->> (rx/from added)
(rx/map process-wasm-object)))))))
(when render-wasm?
(let [local-commits-s
(->> stream
(rx/filter dch/commit?)
(rx/map deref)
(rx/filter #(and (= :local (:source %))
(not (contains? (:tags %) :position-data))))
(rx/filter (complement empty?)))
notifier-s
(rx/merge
(->> local-commits-s (rx/debounce 1000))
(->> stream (rx/filter dps/force-persist?)))
objects-s
(rx/from-atom refs/workspace-page-objects {:emit-current-value? true})
current-page-id-s
(rx/from-atom refs/current-page-id {:emit-current-value? true})]
(->> local-commits-s
(rx/buffer-until notifier-s)
(rx/with-latest-from objects-s)
(rx/map
(fn [[commits objects]]
(->> commits
(mapcat :redo-changes)
(filter #(contains? #{:mod-obj :add-obj} (:type %)))
(filter #(cfh/text-shape? objects (:id %)))
(map #(vector
(:id %)
(wasm.api/calculate-position-data (get objects (:id %))))))))
(rx/with-latest-from current-page-id-s)
(rx/map
(fn [[text-position-data page-id]]
(let [changes
(->> text-position-data
(mapv (fn [[id position-data]]
{:type :mod-obj
:id id
:page-id page-id
:operations
[{:type :set
:attr :position-data
:val position-data
:ignore-touched true
:ignore-geometry true}]})))]
(dch/commit-changes
{:redo-changes changes :undo-changes []
:save-undo? false
:tags #{:position-data}})))))))
(->> stream
(rx/filter dch/commit?)
(rx/map deref)

View File

@@ -0,0 +1,288 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
;;
;; High level helpers to turn a shape subtree into a component and
;; replace equivalent subtrees by instances of that component.
(ns app.main.data.workspace.componentize
(:require
[app.common.data :as d]
[app.common.data.macros :as dm]
[app.common.files.changes-builder :as pcb]
[app.common.files.helpers :as cfh]
[app.common.geom.point :as gpt]
[app.common.logic.libraries :as cll]
[app.common.logic.shapes :as cls]
[app.common.types.shape :as cts]
[app.common.uuid :as uuid]
[app.main.data.changes :as dch]
[app.main.data.helpers :as dsh]
[app.main.data.workspace.libraries :as dwl]
[app.main.data.workspace.selection :as dws]
[app.main.data.workspace.shapes :as dwsh]
[app.main.data.workspace.undo :as dwu]
[beicon.v2.core :as rx]
[potok.v2.core :as ptk]))
;; NOTE: We keep this separate from `workspace.libraries` to avoid
;; introducing more complexity in that already big namespace.
(def ^:private instance-structural-keys
"Keys we do NOT want to copy from the original shape when creating a
new component instance. These are identity / structural / component
metadata keys that must be managed by the component system itself."
#{:id
:parent-id
:frame-id
:shapes
;; Component metadata
:component-id
:component-file
:component-root
:main-instance
:remote-synced
:shape-ref
:touched})
(def ^:private instance-geometry-keys
"Geometry-related keys that we *do* want to override per instance when
copying props from an existing subtree to a component instance."
#{:x
:y
:width
:height
:rotation
:flip-x
:flip-y
:selrect
:points
:proportion
:proportion-lock
:transform
:transform-inverse})
(defn- instantiate-similar-subtrees
"Internal helper. Given an atom `id-ref` that will contain the
`component-id`, replace each subtree rooted at the ids in
`similar-ids` by an instance of that component.
The operation is performed in a single undo transaction:
- Instantiate the component once per similar id, roughly at the same
top-left position as the original root.
- Delete the original subtrees.
- Select the main instance plus all the new instances."
[id-ref root-id similar-ids]
(ptk/reify ::instantiate-similar-subtrees
ptk/WatchEvent
(watch [it state _]
(let [component-id @id-ref
similar-ids (vec (or similar-ids []))]
(if (or (uuid/zero? component-id)
(empty? similar-ids))
(rx/empty)
(let [file-id (:current-file-id state)
page (dsh/lookup-page state)
page-id (:id page)
objects (:objects page)
libraries (dsh/lookup-libraries state)
fdata (dsh/lookup-file-data state file-id)
;; Reference subtree: shapes used to build the component.
;; We'll compute per-shape deltas against this subtree so
;; that we only override attributes that actually differ.
ref-subtree-ids (cfh/get-children-ids objects root-id)
ref-all-ids (into [root-id] ref-subtree-ids)
undo-id (js/Symbol)
;; 1) Instantiate component at each similar root position,
;; preserving per-instance overrides (geometry, style, etc.)
[changes new-root-ids]
(reduce
(fn [[changes acc] sid]
(if-let [shape (get objects sid)]
(let [position (gpt/point (:x shape) (:y shape))
;; Remember original parent and index so we can keep
;; the same ordering among the parent's children.
orig-root (get objects sid)
orig-parent-id (:parent-id orig-root)
orig-index (when orig-parent-id
(cfh/get-position-on-parent objects sid))
;; Instantiate a new component instance at the same position
[new-shape changes']
(cll/generate-instantiate-component
(or changes
(-> (pcb/empty-changes it page-id)
(pcb/with-objects objects)))
objects
file-id
component-id
position
page
libraries)
;; Build a structural mapping between the original subtree
;; (rooted at `sid`) and the new instance subtree.
;; NOTE 1: instantiating a component can introduce an extra
;; wrapper frame, so we try to align the original root
;; with the "equivalent" root inside the instance.
;; NOTE 2: by default the instance may be created *inside*
;; the original shape (because of layout / hit-testing).
;; We explicitly move the new instance to the same parent
;; and index as the original root, so that later deletes of
;; the original subtree don't remove the new instances and
;; the ordering among siblings is preserved.
changes' (cond-> changes'
(some? orig-parent-id)
(pcb/change-parent orig-parent-id [new-shape] orig-index
{:allow-altering-copies true
:ignore-touched true}))
objects' (pcb/get-objects changes')
orig-root (get objects sid)
new-root new-shape
orig-type (:type orig-root)
new-type (:type new-root)
;; Full original subtree (root + descendants)
orig-subtree-ids (cfh/get-children-ids objects sid)
orig-all-ids (into [sid] orig-subtree-ids)
;; Try to find an inner instance root matching the original type
;; when the outer instance root type differs (e.g. rect -> frame+rect).
direct-new-children (cfh/get-children-ids objects' (:id new-root))
candidate-instance-root
(when (and orig-type (not= orig-type new-type))
(let [cands (->> direct-new-children
(filter (fn [nid]
(when-let [s (get objects' nid)]
(= (:type s) orig-type)))))]
(when (= 1 (count cands))
(first cands))))
instance-root-id (or candidate-instance-root (:id new-root))
instance-root (get objects' instance-root-id)
new-subtree-ids (cfh/get-children-ids objects' instance-root-id)
new-all-ids (into [instance-root-id] new-subtree-ids)
id-pairs (map vector orig-all-ids new-all-ids)
changes''
;; Compute per-shape deltas against the reference
;; subtree (root-id) and apply only the differences
;; to the new instance subtree, so we don't blindly
;; overwrite attributes that are the same.
(reduce
(fn [ch [idx orig-id new-id]]
(let [ref-id (nth ref-all-ids idx nil)
ref-shape (get objects ref-id)
orig-shape (get objects orig-id)]
(if (and ref-shape orig-shape)
(let [;; Style / layout / text props (see `extract-props`)
ref-style (cts/extract-props ref-shape)
orig-style (cts/extract-props orig-shape)
style-delta (reduce (fn [m k]
(let [rv (get ref-style k ::none)
ov (get orig-style k ::none)]
(if (= rv ov)
m
(assoc m k ov))))
{}
(keys orig-style))
;; Geometry props
ref-geom (select-keys ref-shape instance-geometry-keys)
orig-geom (select-keys orig-shape instance-geometry-keys)
geom-delta (reduce (fn [m k]
(let [rv (get ref-geom k ::none)
ov (get orig-geom k ::none)]
(if (= rv ov)
m
(assoc m k ov))))
{}
(keys orig-geom))
;; Text content: if the subtree reference and the
;; original differ in `:content`, treat the whole
;; content tree as an override for this instance.
content-delta? (not= (:content ref-shape) (:content orig-shape))]
(-> ch
;; First patch style/text/layout props using the
;; canonical helpers so we don't touch structural ids.
(cond-> (seq style-delta)
(pcb/update-shapes
[new-id]
(fn [s objs] (cts/patch-props s style-delta objs))
{:with-objects? true}))
;; Then patch geometry directly on the instance.
(cond-> (seq geom-delta)
(pcb/update-shapes
[new-id]
(d/patch-object geom-delta)))
;; Finally, if text content differs between the
;; reference subtree and the similar subtree,
;; override the instance content with the original.
(cond-> content-delta?
(pcb/update-shapes
[new-id]
#(assoc % :content (:content orig-shape))))))
ch)))
changes'
(map-indexed (fn [idx [orig-id new-id]]
[idx orig-id new-id])
id-pairs))]
[changes'' (conj acc (:id new-shape))])
;; If the shape does not exist we just skip it
[changes acc]))
[nil []]
similar-ids)
changes (or changes
(-> (pcb/empty-changes it page-id)
(pcb/with-objects objects)))
;; 2) Delete original similar subtrees
;; NOTE: `d/ordered-set` with a single arg treats it as a single
;; element, so we must use `into` when we already have a collection.
ids-to-delete (into (d/ordered-set) similar-ids)
[all-parents changes]
(cls/generate-delete-shapes
changes
fdata
page
objects
ids-to-delete
{:allow-altering-copies true})
;; 3) Select main instance + new instances
;; Root id is kept as-is; add all new roots.
sel-ids (into (d/ordered-set) (cons root-id new-root-ids))]
(rx/of
(dwu/start-undo-transaction undo-id)
(dch/commit-changes changes)
(ptk/data-event :layout/update {:ids all-parents})
(dwu/commit-undo-transaction undo-id))))))))
(defn componentize-similar-subtrees
"Turn the subtree rooted at `root-id` into a component, then replace
the subtrees rooted at `similar-ids` with instances of that component.
This is implemented in two phases:
1) Use the existing `dwl/add-component` flow to create a component
from `root-id` (and obtain its `component-id`).
2) Using the new `component-id`, instantiate the component once per
entry in `similar-ids` and delete the old subtrees."
[root-id similar-ids]
(dm/assert!
"expected valid uuid for `root-id`"
(uuid? root-id))
(let [similar-ids (vec (or similar-ids []))]
(ptk/reify ::componentize-similar-subtrees
ptk/WatchEvent
(watch [_ _ _]
(let [id-ref (atom uuid/zero)]
(rx/concat
;; 1) Create component using the existing pipeline
(rx/of (dwl/add-component id-ref (d/ordered-set root-id)))
;; 2) Replace similar subtrees by instances of the new component
(rx/of (instantiate-similar-subtrees id-ref root-id similar-ids))))))))

View File

@@ -30,6 +30,9 @@
(def profile
(l/derived (l/key :profile) st/state))
(def current-page-id
(l/derived (l/key :current-page-id) st/state))
(def team
(l/derived (fn [state]
(let [team-id (:current-team-id state)

View File

@@ -60,6 +60,7 @@
current-id (get state :id)
current-value (get state :current-value)
current-label (get label-index current-value)
is-open? (get state :is-open?)
node-ref (mf/use-ref nil)

View File

@@ -45,7 +45,7 @@
.element-list {
@include t.use-typography("body-large");
color: var(--modal-text-foreground-color);
overflow-y: scroll;
overflow-y: auto;
margin-block: 0;
}

View File

@@ -5,6 +5,7 @@
// Copyright (c) KALEIDOS INC
@use "refactor/common-refactor.scss" as deprecated;
@use "ds/_utils.scss" as *;
.layer-row {
--layer-indentation-size: calc(#{deprecated.$s-4} * 6);
@@ -87,7 +88,7 @@
height: deprecated.$s-32;
width: calc(100% - (var(--depth) * var(--layer-indentation-size)));
cursor: pointer;
min-width: px2rem(140);
&.filtered {
width: calc(100% - deprecated.$s-12);
}

View File

@@ -211,9 +211,7 @@
overflow-x: auto;
overflow-y: overlay;
scrollbar-gutter: stable;
.element-list {
width: var(--left-sidebar-width);
display: grid;
}
}
.element-list {
display: grid;
}

View File

@@ -102,7 +102,7 @@
[:> deprecated-input/numeric-input*
{:placeholder (cond
(not all-equal?)
"Mixed"
(tr "settings.multiple")
(= :multiple (:r1 values))
(tr "settings.multiple")
:else

View File

@@ -265,11 +265,13 @@
(mf/deps font on-change)
(fn [new-variant-id]
(let [variant (d/seek #(= new-variant-id (:id %)) (:variants font))]
(on-change {:font-id (:id font)
:font-family (:family font)
:font-variant-id new-variant-id
:font-weight (:weight variant)
:font-style (:style variant)})
(when-not (nil? variant)
(on-change {:font-id (:id font)
:font-family (:family font)
:font-variant-id new-variant-id
:font-weight (:weight variant)
:font-style (:style variant)}))
(dom/blur! (dom/get-target new-variant-id)))))
on-font-select
@@ -342,12 +344,13 @@
{:value (:id variant)
:key (pr-str variant)
:label (:name variant)})))
variant-options (if (= font-size :multiple)
variant-options (if (= font-variant-id :multiple)
(conj basic-variant-options
{:value :multiple
{:value ""
:key :multiple-variants
:label "--"})
basic-variant-options)]
;; TODO Add disabled mode
[:& select
{:class (stl/css :font-variant-select)

View File

@@ -68,7 +68,7 @@
(mf/defc color-token-row*
{::mf/private true}
[{:keys [active-tokens color-token color on-swatch-click-token detach-token open-modal-from-token]}]
[{:keys [active-tokens applied-token-name color on-swatch-click-token detach-token open-modal-from-token]}]
(let [;; `active-tokens` may be provided as a `delay` (lazy computation).
;; In that case we must deref it (`@active-tokens`) to force evaluation
;; and obtain the actual value. If its already realized (not a delay),
@@ -77,21 +77,22 @@
@active-tokens
active-tokens)
color-tokens (:color active-tokens)
active-color-tokens (:color active-tokens)
token (some #(when (= (:name %) color-token) %) color-tokens)
token (some #(when (= (:name %) applied-token-name) %) active-color-tokens)
on-detach-token
(mf/use-fn
(mf/deps detach-token token color-token)
(mf/deps detach-token token applied-token-name)
(fn []
(let [token (or token color-token)]
(let [token (or token applied-token-name)]
(detach-token token))))
has-errors (some? (:errors token))
token-name (:name token)
resolved (:resolved-value token)
not-active (and (some? active-tokens) (nil? token))
not-active (and (empty? active-tokens)
(nil? token))
id (dm/str (:id token) "-name")
swatch-tooltip-content (cond
not-active
@@ -109,7 +110,7 @@
#(mf/html
[:div
[:span (dm/str (tr "workspace.tokens.token-name") ": ")]
[:span {:class (stl/css :token-name-tooltip)} color-token]]))]
[:span {:class (stl/css :token-name-tooltip)} applied-token-name]]))]
[:div {:class (stl/css :color-info)}
[:div {:class (stl/css-case :token-color-wrapper true
@@ -128,7 +129,7 @@
:class (stl/css :token-tooltip)}
[:div {:class (stl/css :token-name)
:aria-labelledby id}
(or token-name color-token)]]
(or token-name applied-token-name)]]
[:div {:class (stl/css :token-actions)}
[:> icon-button*
{:variant "action"
@@ -146,7 +147,11 @@
on-change on-reorder on-detach on-open on-close on-remove origin on-detach-token
disable-drag on-focus on-blur select-only select-on-focus on-token-change applied-token]}]
(let [token-color (contains? cfg/flags :token-color)
(let [;; TODO: Remove this workaround fixing `get-attrs*` fn on sidebar/options/shapes/multiple.cljs
applied-token (if (= :multiple applied-token)
nil
applied-token)
token-color (contains? cfg/flags :token-color)
libraries (mf/deref refs/files)
color-without-hash (mf/use-memo
@@ -177,7 +182,6 @@
(-> (deref active-tokens*)
(select-keys (get tk/tokens-by-input origin))
(not-empty)))))
on-focus'
(mf/use-fn
(mf/deps on-focus)
@@ -352,7 +356,7 @@
(cond
(and token-color applied-token)
[:> color-token-row* {:active-tokens tokens
:color-token applied-token
:applied-token-name applied-token
:color (dissoc color :ref-id :ref-file)
:on-swatch-click-token on-swatch-click-token
:detach-token detach-token

View File

@@ -63,7 +63,8 @@
:data {:index index})
[nil nil])
stroke-color-token (:stroke-color applied-tokens)
stroke-color-token
(:stroke-color applied-tokens)
on-color-change-refactor
(mf/use-fn

View File

@@ -13,11 +13,16 @@
[app.common.geom.shapes :as gsh]
[app.common.types.color :as clr]
[app.common.types.component :as ctk]
[app.common.types.container :as ctn]
[app.common.types.path :as path]
[app.common.types.shape :as cts]
[app.common.types.shape-tree :as ctt]
[app.common.types.shape.layout :as ctl]
[app.config :as cf]
[app.graph-wasm.api :as graph-wasm.api]
[app.main.data.workspace.componentize :as dwc]
[app.main.data.workspace.modifiers :as dwm]
[app.main.data.workspace.selection :as dws]
[app.main.data.workspace.variants :as dwv]
[app.main.features :as features]
[app.main.refs :as refs]
@@ -57,8 +62,11 @@
[app.main.ui.workspace.viewport.utils :as utils]
[app.main.ui.workspace.viewport.viewport-ref :refer [create-viewport-ref]]
[app.main.ui.workspace.viewport.widgets :as widgets]
[app.main.worker :as worker]
[app.util.debug :as dbg]
[app.util.modules :as mod]
[beicon.v2.core :as rx]
[promesa.core :as p]
[rumext.v2 :as mf]))
;; --- Viewport
@@ -134,6 +142,7 @@
mod? (mf/use-state false)
space? (mf/use-state false)
z? (mf/use-state false)
g? (mf/use-state false)
cursor (mf/use-state #(utils/get-cursor :pointer-inner))
hover-ids (mf/use-state nil)
hover (mf/use-state nil)
@@ -302,12 +311,79 @@
(mf/use-fn
(mf/deps first-shape)
#(st/emit!
(dwv/add-new-variant (:id first-shape))))]
(dwv/add-new-variant (:id first-shape))))
graph-wasm-enabled? (features/use-feature "graph-wasm/v1")]
(mf/with-effect [page-id]
(when graph-wasm-enabled?
;; Initialize graph-wasm in the worker to avoid blocking main thread
(let [subscription
(->> (worker/ask! {:cmd :graph-wasm/init})
(rx/filter #(= (:status %) :ok))
(rx/take 1)
(rx/merge-map (fn [_]
(worker/ask! {:cmd :graph-wasm/set-objects
:objects base-objects}))))]
(rx/subscribe subscription
(fn [result]
(when (= (:status result) :ok)
(js/console.debug "Graph WASM initialized in worker"
(select-keys result [:processed]))))
(fn [error]
(js/console.error "Error initializing graph-wasm in worker:" error))
(fn []
(js/console.debug "Graph WASM worker operations completed"))))))
(mf/with-effect [selected @g?]
(when graph-wasm-enabled?
;; Search for similar shapes when selection changes or when
;; the user presses the \"c\" key while having a single
;; selection.
(when (and @g?
(some? selected)
(= (count selected) 1))
(let [selected-id (first selected)
selected-shape (get base-objects selected-id)
;; Skip shapes that already belong to a component
non-component? (and (some? selected-shape)
(not (ctn/in-any-component? base-objects selected-shape)))]
(println selected-shape)
(println (ctn/in-any-component? base-objects selected-shape))
(when non-component?
(let [subscription
(worker/ask! {:cmd :graph-wasm/search-similar-shapes
:shape-id selected-id})]
(rx/subscribe subscription
(fn [result]
(when (= (:status result) :ok)
(let [raw-similar-shapes (:similar-shapes result)
;; Filter out shapes that already belong to some component
;; (main instance, instance head or inside a component copy).
similar-shapes (->> raw-similar-shapes
(remove (fn [sid]
(when-let [s (get base-objects sid)]
(ctn/in-any-component? base-objects s))))
(into []))]
(when (d/not-empty? similar-shapes)
;; Transform the selected subtree into a component and
;; replace similar subtrees by instances of that component.
(st/emit! (dwc/componentize-similar-subtrees
selected-id
similar-shapes))))))
(fn [error]
(js/console.error "Error searching similar shapes:" error))
(fn []
(js/console.debug "Similar shapes search completed")))))))))
(hooks/setup-dom-events zoom disable-paste-ref in-viewport-ref read-only? drawing-tool path-drawing?)
(hooks/setup-viewport-size vport viewport-ref)
(hooks/setup-cursor cursor alt? mod? space? panning drawing-tool path-drawing? path-editing? z? read-only?)
(hooks/setup-keyboard alt? mod? space? z? shift?)
(hooks/setup-keyboard alt? mod? space? z? shift? g?)
(hooks/setup-hover-shapes page-id move-stream base-objects transform selected mod? hover measure-hover
hover-ids hover-top-frame-id @hover-disabled? focus zoom show-measures?)
(hooks/setup-viewport-modifiers modifiers base-objects)

View File

@@ -16,6 +16,7 @@
[app.common.geom.shapes.points :as gpo]
[app.common.types.shape.layout :as ctl]
[app.common.uuid :as uuid]
[app.render-wasm.api :as wasm.api]
[cuerdas.core :as str]
[rumext.v2 :as mf]))
@@ -275,3 +276,26 @@
:y2 (:y end-p)
:style {:stroke "red"
:stroke-width (/ 1 zoom)}}]))]))))
(mf/defc debug-text-wasm-position-data
{::mf/wrap-props false}
[props]
(let [zoom (unchecked-get props "zoom")
selected-shapes (unchecked-get props "selected-shapes")
selected-text
(when (and (= (count selected-shapes) 1) (= :text (-> selected-shapes first :type)))
(first selected-shapes))
position-data
(when selected-text
(wasm.api/calculate-position-data selected-text))]
(for [{:keys [x y width height]} position-data]
[:rect {:x x
:y (- y height)
:width width
:height height
:fill "none"
:strokeWidth (/ 1 zoom)
:stroke "red"}])))

View File

@@ -124,7 +124,7 @@
(reset! cursor new-cursor))))))
(defn setup-keyboard
[alt* mod* space* z* shift*]
[alt* mod* space* z* shift* g*]
(let [kbd-zoom-s
(mf/with-memo []
(->> ms/keyboard
@@ -151,12 +151,22 @@
(rx/filter kbd/z?)
(rx/filter (complement kbd/editing-event?))
(rx/map kbd/key-down-event?)
(rx/pipe (rxo/distinct-contiguous))))]
(rx/pipe (rxo/distinct-contiguous))))
kbd-g-s
(mf/with-memo []
(let [c-pred (kbd/is-key-ignore-case? "g")]
(->> ms/keyboard
(rx/filter c-pred)
(rx/filter (complement kbd/editing-event?))
(rx/map kbd/key-down-event?)
(rx/pipe (rxo/distinct-contiguous)))))]
(hooks/use-stream ms/keyboard-alt (partial reset! alt*))
(hooks/use-stream ms/keyboard-space (partial reset! space*))
(hooks/use-stream kbd-z-s (partial reset! z*))
(hooks/use-stream kbd-shift-s (partial reset! shift*))
(hooks/use-stream kbd-g-s (partial reset! g*))
(hooks/use-stream ms/keyboard-mod
(fn [value]
(reset! mod* value)

View File

@@ -12,10 +12,12 @@
[app.common.files.helpers :as cfh]
[app.common.geom.shapes :as gsh]
[app.common.types.color :as clr]
[app.common.types.component :as ctk]
[app.common.types.path :as path]
[app.common.types.shape :as cts]
[app.common.types.shape.layout :as ctl]
[app.main.data.workspace.transforms :as dwt]
[app.main.data.workspace.variants :as dwv]
[app.main.features :as features]
[app.main.refs :as refs]
[app.main.store :as st]
@@ -120,6 +122,7 @@
mod? (mf/use-state false)
space? (mf/use-state false)
z? (mf/use-state false)
c? (mf/use-state false)
cursor (mf/use-state (utils/get-cursor :pointer-inner))
hover-ids (mf/use-state nil)
hover (mf/use-state nil)
@@ -257,6 +260,16 @@
first-shape (first selected-shapes)
show-add-variant? (and single-select?
(or (ctk/is-variant-container? first-shape)
(ctk/is-variant? first-shape)))
add-variant
(mf/use-fn
(mf/deps first-shape)
#(st/emit!
(dwv/add-new-variant (:id first-shape))))
show-padding?
(and (nil? transform)
single-select?
@@ -348,7 +361,7 @@
(hooks/setup-dom-events zoom disable-paste-ref in-viewport-ref read-only? drawing-tool path-drawing?)
(hooks/setup-viewport-size vport viewport-ref)
(hooks/setup-cursor cursor alt? mod? space? panning drawing-tool path-drawing? path-editing? z? read-only?)
(hooks/setup-keyboard alt? mod? space? z? shift?)
(hooks/setup-keyboard alt? mod? space? z? shift? c?)
(hooks/setup-hover-shapes page-id move-stream base-objects transform selected mod? hover measure-hover
hover-ids hover-top-frame-id @hover-disabled? focus zoom show-measures?)
(hooks/setup-shortcuts path-editing? path-drawing? text-editing? grid-editing?)
@@ -635,6 +648,12 @@
:hover-top-frame-id @hover-top-frame-id
:zoom zoom}])
(when (dbg/enabled? :text-outline)
[:& wvd/debug-text-wasm-position-data
{:selected-shapes selected-shapes
:objects base-objects
:zoom zoom}])
(when show-selection-handlers?
[:g.selection-handlers {:clipPath "url(#clip-handlers)"}
(when-not text-editing?
@@ -663,6 +682,11 @@
{:id (first selected)
:zoom zoom}])
(when show-add-variant?
[:> widgets/button-add* {:shape first-shape
:zoom zoom
:on-click add-variant}])
[:g.grid-layout-editor {:clipPath "url(#clip-handlers)"}
(when show-grid-editor?
[:& grid-layout/editor

View File

@@ -23,6 +23,7 @@
[app.main.refs :as refs]
[app.main.render :as render]
[app.main.store :as st]
[app.main.ui.shapes.text]
[app.main.worker :as mw]
[app.render-wasm.api.fonts :as f]
[app.render-wasm.api.texts :as t]
@@ -33,7 +34,7 @@
[app.render-wasm.performance :as perf]
[app.render-wasm.serializers :as sr]
[app.render-wasm.serializers.color :as sr-clr]
[app.render-wasm.svg-fills :as svg-fills]
[app.render-wasm.svg-filters :as svg-filters]
;; FIXME: rename; confunsing name
[app.render-wasm.wasm :as wasm]
[app.util.debug :as dbg]
@@ -42,6 +43,7 @@
[app.util.modules :as mod]
[app.util.text.content :as tc]
[beicon.v2.core :as rx]
[cuerdas.core :as str]
[promesa.core :as p]
[rumext.v2 :as mf]))
@@ -703,7 +705,7 @@
(set-grid-layout-columns (get shape :layout-grid-columns))
(set-grid-layout-cells (get shape :layout-grid-cells)))
(defn set-layout-child
(defn set-layout-data
[shape]
(let [margins (get shape :layout-item-margin)
margin-top (get margins :m1 0)
@@ -726,7 +728,7 @@
is-absolute (boolean (get shape :layout-item-absolute))
z-index (get shape :layout-item-z-index)]
(h/call wasm/internal-module
"_set_layout_child_data"
"_set_layout_data"
margin-top
margin-right
margin-bottom
@@ -746,6 +748,11 @@
is-absolute
(d/nilv z-index 0))))
(defn has-any-layout-prop? [shape]
(some #(and (keyword? %)
(str/starts-with? (name %) "layout-"))
(keys shape)))
(defn clear-layout
[]
(h/call wasm/internal-module "_clear_shape_layout"))
@@ -753,10 +760,10 @@
(defn- set-shape-layout
[shape objects]
(clear-layout)
(when (or (ctl/any-layout? shape)
(ctl/any-layout-immediate-child? objects shape))
(set-layout-child shape))
(ctl/any-layout-immediate-child? objects shape)
(has-any-layout-prop? shape))
(set-layout-data shape))
(when (ctl/flex-layout? shape)
(set-flex-layout shape))
@@ -875,27 +882,43 @@
(def render-finish
(letfn [(do-render [ts]
(perf/begin-measure "render-finish")
(h/call wasm/internal-module "_set_view_end")
(render ts))]
(render ts)
(perf/end-measure "render-finish"))]
(fns/debounce do-render DEBOUNCE_DELAY_MS)))
(def render-pan
(fns/throttle render THROTTLE_DELAY_MS))
(letfn [(do-render-pan [ts]
(perf/begin-measure "render-pan")
(render ts)
(perf/end-measure "render-pan"))]
(fns/throttle do-render-pan THROTTLE_DELAY_MS)))
(defn set-view-box
[prev-zoom zoom vbox]
(h/call wasm/internal-module "_set_view" zoom (- (:x vbox)) (- (:y vbox)))
(let [is-pan (mth/close? prev-zoom zoom)]
(perf/begin-measure "set-view-box")
(h/call wasm/internal-module "_set_view_start")
(h/call wasm/internal-module "_set_view" zoom (- (:x vbox)) (- (:y vbox)))
(if (mth/close? prev-zoom zoom)
(do (render-pan)
(render-finish))
(do (h/call wasm/internal-module "_render_from_cache" 0)
(render-finish))))
(if is-pan
(do (perf/end-measure "set-view-box")
(perf/begin-measure "set-view-box::pan")
(render-pan)
(render-finish)
(perf/end-measure "set-view-box::pan"))
(do (perf/end-measure "set-view-box")
(perf/begin-measure "set-view-box::zoom")
(h/call wasm/internal-module "_render_from_cache" 0)
(render-finish)
(perf/end-measure "set-view-box::zoom")))))
(defn set-object
[objects shape]
(perf/begin-measure "set-object")
(let [id (dm/get-prop shape :id)
(let [shape (svg-filters/apply-svg-derived shape)
id (dm/get-prop shape :id)
type (dm/get-prop shape :type)
parent-id (get shape :parent-id)
@@ -909,14 +932,7 @@
rotation (get shape :rotation)
transform (get shape :transform)
;; If the shape comes from an imported SVG (we know this because
;; it has the :svg-attrs attribute) and it does not have its
;; own fill, we set a default black fill. This fill will be
;; inherited by child nodes and emulates the behavior of
;; standard SVG, where a node without an explicit fill
;; defaults to black.
fills (svg-fills/resolve-shape-fills shape)
fills (get shape :fills)
strokes (if (= type :group)
[] (get shape :strokes))
children (get shape :shapes)
@@ -960,12 +976,11 @@
(set-shape-svg-attrs svg-attrs))
(when (and (some? content) (= type :svg-raw))
(set-shape-svg-raw-content (get-static-markup shape)))
(when (some? shadows) (set-shape-shadows shadows))
(set-shape-shadows shadows)
(when (= type :text)
(set-shape-grow-type grow-type))
(set-shape-layout shape objects)
(set-shape-selrect selrect)
(let [pending_thumbnails (into [] (concat
@@ -989,10 +1004,7 @@
(run!
(fn [id]
(f/update-text-layout id)
(mw/emit! {:cmd :index/update-text-rect
:page-id (:current-page-id @st/state)
:shape-id id
:dimensions (get-text-dimensions id)})))))
(update-text-rect! id)))))
(defn process-pending
([shapes thumbnails full on-complete]
@@ -1233,6 +1245,8 @@
(when-not (nil? context)
(let [handle (.registerContext ^js gl context #js {"majorVersion" 2})]
(.makeContextCurrent ^js gl handle)
(set! wasm/gl-context-handle handle)
(set! wasm/gl-context context)
;; Force the WEBGL_debug_renderer_info extension as emscripten does not enable it
(.getExtension context "WEBGL_debug_renderer_info")
@@ -1255,6 +1269,20 @@
(set! wasm/context-initialized? false)
(h/call wasm/internal-module "_clean_up")
;; Ensure the WebGL context is properly disposed so browsers do not keep
;; accumulating active contexts between page switches.
(when-let [gl (unchecked-get wasm/internal-module "GL")]
(when-let [handle wasm/gl-context-handle]
(try
;; Ask the browser to release resources explicitly if available.
(when-let [ctx wasm/gl-context]
(when-let [lose-ext (.getExtension ^js ctx "WEBGL_lose_context")]
(.loseContext ^js lose-ext)))
(.deleteContext ^js gl handle)
(finally
(set! wasm/gl-context-handle nil)
(set! wasm/gl-context nil)))))
;; If this calls panics we don't want to crash. This happens sometimes
;; with hot-reload in develop
(catch :default error
@@ -1348,6 +1376,59 @@
(h/call wasm/internal-module "_end_temp_objects")
content)))
(def POSITION-DATA-U8-SIZE 36)
(def POSITION-DATA-U32-SIZE (/ POSITION-DATA-U8-SIZE 4))
(defn calculate-position-data
[shape]
(when wasm/context-initialized?
(use-shape (:id shape))
(let [heapf32 (mem/get-heap-f32)
heapu32 (mem/get-heap-u32)
offset (-> (h/call wasm/internal-module "_calculate_position_data")
(mem/->offset-32))
length (aget heapu32 offset)
max-offset (+ offset 1 (* length POSITION-DATA-U32-SIZE))
result
(loop [result (transient [])
offset (inc offset)]
(if (< offset max-offset)
(let [entry (dr/read-position-data-entry heapu32 heapf32 offset)]
(recur (conj! result entry)
(+ offset POSITION-DATA-U32-SIZE)))
(persistent! result)))
result
(->> result
(mapv
(fn [{:keys [paragraph span start-pos end-pos direction x y width height]}]
(let [content (:content shape)
element (-> content :children
(get 0) :children ;; paragraph-set
(get paragraph) :children ;; paragraph
(get span))
text (subs (:text element) start-pos end-pos)]
{:x x
:y (+ y height)
:width width
:height height
:direction (dr/translate-direction direction)
:font-family (get element :font-family)
:font-size (get element :font-size)
:font-weight (get element :font-weight)
:text-transform (get element :text-transform)
:text-decoration (get element :text-decoration)
:letter-spacing (get element :letter-spacing)
:font-style (get element :font-style)
:fills (get element :fills)
:text text}))))]
(mem/free)
result)))
(defn init-wasm-module
[module]
(let [default-fn (unchecked-get module "default")

View File

@@ -45,4 +45,29 @@
:center (gpt/point cx cy)
:transform (gmt/matrix a b c d e f)}))
(defn read-position-data-entry
[heapu32 heapf32 offset]
(let [paragraph (aget heapu32 (+ offset 0))
span (aget heapu32 (+ offset 1))
start-pos (aget heapu32 (+ offset 2))
end-pos (aget heapu32 (+ offset 3))
x (aget heapf32 (+ offset 4))
y (aget heapf32 (+ offset 5))
width (aget heapf32 (+ offset 6))
height (aget heapf32 (+ offset 7))
direction (aget heapu32 (+ offset 8))]
{:paragraph paragraph
:span span
:start-pos start-pos
:end-pos end-pos
:x x
:y y
:width width
:height height
:direction direction}))
(defn translate-direction
[direction]
(case direction
0 "rtl"
"ltr"))

View File

@@ -14,7 +14,7 @@
[app.common.types.shape.layout :as ctl]
[app.main.refs :as refs]
[app.render-wasm.api :as api]
[app.render-wasm.svg-fills :as svg-fills]
[app.render-wasm.svg-filters :as svg-filters]
[app.render-wasm.wasm :as wasm]
[beicon.v2.core :as rx]
[cljs.core :as c]
@@ -130,7 +130,11 @@
(defn- set-wasm-attr!
[shape k]
(when wasm/context-initialized?
(let [v (get shape k)
(let [shape (case k
:svg-attrs (svg-filters/apply-svg-derived (assoc shape :svg-attrs (get shape :svg-attrs)))
(:fills :blur :shadow) (svg-filters/apply-svg-derived shape)
shape)
v (get shape k)
id (get shape :id)]
(case k
:parent-id
@@ -163,8 +167,7 @@
(api/set-shape-transform v)
:fills
(let [fills (svg-fills/resolve-shape-fills shape)]
(into [] (api/set-shape-fills id fills false)))
(api/set-shape-fills id v false)
:strokes
(into [] (api/set-shape-strokes id v false))
@@ -222,8 +225,12 @@
v])
:svg-attrs
(when (cfh/path-shape? shape)
(api/set-shape-svg-attrs v))
(do
(api/set-shape-svg-attrs v)
;; Always update fills/blur/shadow to clear previous state if filters disappear
(api/set-shape-fills id (:fills shape) false)
(api/set-shape-blur (:blur shape))
(api/set-shape-shadows (:shadow shape)))
:masked-group
(when (cfh/mask-shape? shape)
@@ -262,7 +269,7 @@
:layout-item-min-w
:layout-item-absolute
:layout-item-z-index)
(api/set-layout-child shape)
(api/set-layout-data shape)
:layout-grid-rows
(api/set-grid-layout-rows v)
@@ -292,7 +299,7 @@
(ctl/flex-layout? shape)
(api/set-flex-layout shape))
(api/set-layout-child shape))
(api/set-layout-data shape))
;; Property not in WASM
nil))))

View File

@@ -74,6 +74,30 @@
:width (max 0.01 (or (dm/get-prop shape :width) 1))
:height (max 0.01 (or (dm/get-prop shape :height) 1))}))))
(defn- apply-svg-transform
"Applies SVG transform to a point if present."
[pt svg-transform]
(if svg-transform
(gpt/transform pt svg-transform)
pt))
(defn- apply-viewbox-transform
"Transforms a point from viewBox space to selrect space."
[pt viewbox rect]
(if viewbox
(let [{svg-x :x svg-y :y svg-width :width svg-height :height} viewbox
rect-width (max 0.01 (dm/get-prop rect :width))
rect-height (max 0.01 (dm/get-prop rect :height))
origin-x (or (dm/get-prop rect :x) (dm/get-prop rect :x1) 0)
origin-y (or (dm/get-prop rect :y) (dm/get-prop rect :y1) 0)
scale-x (/ rect-width svg-width)
scale-y (/ rect-height svg-height)
;; Transform from viewBox space to selrect space
transformed-x (+ origin-x (* (- (dm/get-prop pt :x) svg-x) scale-x))
transformed-y (+ origin-y (* (- (dm/get-prop pt :y) svg-y) scale-y))]
(gpt/point transformed-x transformed-y))
pt))
(defn- normalize-point
[pt units shape]
(if (= units "userspaceonuse")
@@ -81,9 +105,16 @@
width (max 0.01 (dm/get-prop rect :width))
height (max 0.01 (dm/get-prop rect :height))
origin-x (or (dm/get-prop rect :x) (dm/get-prop rect :x1) 0)
origin-y (or (dm/get-prop rect :y) (dm/get-prop rect :y1) 0)]
(gpt/point (/ (- (dm/get-prop pt :x) origin-x) width)
(/ (- (dm/get-prop pt :y) origin-y) height)))
origin-y (or (dm/get-prop rect :y) (dm/get-prop rect :y1) 0)
svg-transform (:svg-transform shape)
viewbox (:svg-viewbox shape)
;; For userSpaceOnUse, coordinates are in SVG user space
;; We need to transform them to shape space before normalizing
pt-after-svg-transform (apply-svg-transform pt svg-transform)
transformed-pt (apply-viewbox-transform pt-after-svg-transform viewbox rect)
normalized-x (/ (- (dm/get-prop transformed-pt :x) origin-x) width)
normalized-y (/ (- (dm/get-prop transformed-pt :y) origin-y) height)]
(gpt/point normalized-x normalized-y))
pt))
(defn- normalize-attrs
@@ -257,18 +288,25 @@
(parse-gradient-stop node))))
vec)]
(when (seq stops)
(let [[center radius-point]
(let [[center point-x point-y]
(let [points (apply-gradient-transform [(gpt/point cx cy)
(gpt/point (+ cx r) cy)]
(gpt/point (+ cx r) cy)
(gpt/point cx (+ cy r))]
transform)]
(map #(normalize-point % units shape) points))
radius (gpt/distance center radius-point)]
radius-x (gpt/distance center point-x)
radius-y (gpt/distance center point-y)
;; Prefer Y as the base radius so width becomes the X/Y ratio.
base-radius (if (pos? radius-y) radius-y radius-x)
radius-point (if (pos? radius-y) point-y point-x)
width (let [safe-radius (max base-radius 1.0e-6)]
(/ radius-x safe-radius))]
{:type :radial
:start-x (dm/get-prop center :x)
:start-y (dm/get-prop center :y)
:end-x (dm/get-prop radius-point :x)
:end-y (dm/get-prop radius-point :y)
:width radius
:width width
:stops stops}))))
(defn- svg-gradient->fill

View File

@@ -0,0 +1,98 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns app.render-wasm.svg-filters
(:require
[app.common.data :as d]
[app.common.data.macros :as dm]
[app.common.svg :as csvg]
[app.common.uuid :as uuid]
[app.render-wasm.svg-fills :as svg-fills]))
(def ^:private drop-shadow-tags
#{:feOffset :feGaussianBlur :feColorMatrix})
(defn- find-filter-element
"Finds a filter element by tag in filter content."
[filter-content tag]
(some #(when (= tag (:tag %)) %) filter-content))
(defn- find-filter-def
[shape]
(let [filter-attr (or (dm/get-in shape [:svg-attrs :filter])
(dm/get-in shape [:svg-attrs :style :filter]))
svg-defs (dm/get-prop shape :svg-defs)]
(when (and filter-attr svg-defs)
(let [filter-ids (csvg/extract-ids filter-attr)]
(some #(get svg-defs %) filter-ids)))))
(defn- build-blur
[gaussian-blur]
(when gaussian-blur
{:id (uuid/next)
:type :layer-blur
;; For layer blur the value matches stdDeviation directly
:value (-> (dm/get-in gaussian-blur [:attrs :stdDeviation])
(d/parse-double 0))
:hidden false}))
(defn- build-drop-shadow
[filter-content drop-shadow-elements]
(let [offset-elem (find-filter-element filter-content :feOffset)]
(when (and offset-elem (seq drop-shadow-elements))
(let [blur-elem (find-filter-element drop-shadow-elements :feGaussianBlur)
dx (-> (dm/get-in offset-elem [:attrs :dx])
(d/parse-double 0))
dy (-> (dm/get-in offset-elem [:attrs :dy])
(d/parse-double 0))
blur-value (if blur-elem
(-> (dm/get-in blur-elem [:attrs :stdDeviation])
(d/parse-double 0)
(* 2))
0)]
[{:id (uuid/next)
:style :drop-shadow
:offset-x dx
:offset-y dy
:blur blur-value
:spread 0
:hidden false
;; TODO: parse feColorMatrix to extract color/opacity
:color {:color "#000000" :opacity 1}}]))))
(defn apply-svg-filters
"Derives native blur/shadow from SVG filter definitions when the shape does
not already have them. The SVG attributes are left untouched so SVG fallback
rendering keeps working the same way as gradient fills."
[shape]
(let [existing-blur (:blur shape)
existing-shadow (:shadow shape)]
(if-let [filter-def (find-filter-def shape)]
(let [content (:content filter-def)
gaussian-blur (find-filter-element content :feGaussianBlur)
drop-shadow-elements (filter #(contains? drop-shadow-tags (:tag %)) content)
blur (or existing-blur (build-blur gaussian-blur))
shadow (if (seq existing-shadow)
existing-shadow
(build-drop-shadow content drop-shadow-elements))]
(cond-> shape
blur (assoc :blur blur)
(seq shadow) (assoc :shadow shadow)))
shape)))
(defn apply-svg-derived
"Applies SVG-derived effects (fills, blur, shadows) uniformly.
- Keeps user fills if present; otherwise derives from SVG.
- Converts SVG filters into native blur/shadow when needed.
- Always returns shape with :fills (possibly []) and blur/shadow keys."
[shape]
(let [shape' (apply-svg-filters shape)
fills (or (svg-fills/resolve-shape-fills shape') [])]
(assoc shape'
:fills fills
:blur (:blur shape')
:shadow (:shadow shape'))))

View File

@@ -9,6 +9,8 @@
(defonce internal-frame-id nil)
(defonce internal-module #js {})
(defonce gl-context-handle nil)
(defonce gl-context nil)
(defonce serializers
#js {:blur-type shared/RawBlurType
:blend-mode shared/RawBlendMode

View File

@@ -11,6 +11,7 @@
[app.common.schema :as sm]
[app.common.types.objects-map]
[app.util.object :as obj]
[app.worker.graph-wasm]
[app.worker.impl :as impl]
[app.worker.import]
[app.worker.index]

View File

@@ -0,0 +1,181 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns app.worker.graph-wasm
"Graph WASM operations within the worker."
(:require
[app.common.data.macros :as dm]
[app.common.logging :as log]
[app.common.uuid :as uuid]
[app.config :as cf]
[app.graph-wasm.wasm :as wasm]
[app.render-wasm.helpers :as h]
[app.render-wasm.serializers :as sr]
[app.worker.impl :as impl]
[beicon.v2.core :as rx]
[promesa.core :as p]))
(log/set-level! :info)
(defn- use-shape
[module id]
(let [buffer (uuid/get-u32 id)]
(h/call module "_use_shape"
(aget buffer 0)
(aget buffer 1)
(aget buffer 2)
(aget buffer 3))))
(defn- set-shape-parent-id
[module id]
(let [buffer (uuid/get-u32 id)]
(h/call module "_set_shape_parent"
(aget buffer 0)
(aget buffer 1)
(aget buffer 2)
(aget buffer 3))))
(defn- set-shape-type
[module type]
(h/call module "_set_shape_type" (sr/translate-shape-type type)))
(defn- set-shape-selrect
[module selrect]
(h/call module "_set_shape_selrect"
(dm/get-prop selrect :x1)
(dm/get-prop selrect :y1)
(dm/get-prop selrect :x2)
(dm/get-prop selrect :y2)))
(defn- set-object
[module shape]
(let [id (dm/get-prop shape :id)
type (dm/get-prop shape :type)
parent-id (get shape :parent-id)
selrect (get shape :selrect)]
(use-shape module id)
(set-shape-type module type)
(set-shape-parent-id module parent-id)
(set-shape-selrect module selrect)))
(defonce ^:private graph-wasm-module
(delay
(let [module (unchecked-get js/globalThis "GraphWasmModule")
init-fn (unchecked-get module "default")
href (cf/resolve-href "js/graph-wasm.wasm")]
(->> (init-fn #js {:locateFile (constantly href)})
(p/fnly (fn [module cause]
(if cause
(js/console.error cause)
(set! wasm/internal-module module))))))))
(defmethod impl/handler :graph-wasm/init
[message transfer]
(rx/create
(fn [subs]
(-> @graph-wasm-module
(p/then (fn [module]
(if module
(try
(h/call module "_init")
(rx/push! subs {:status :ok})
(rx/end! subs)
(catch :default cause
(log/error :hint "Error in graph-wasm/init" :cause cause)
(rx/error! subs cause)
(rx/end! subs)))
(do
(log/warn :hint "Graph WASM module not available")
(rx/push! subs {:status :error :message "Module not available"})
(rx/end! subs)))))
(p/catch (fn [cause]
(log/error :hint "Error loading graph-wasm module" :cause cause)
(rx/error! subs cause)
(rx/end! subs))))
nil)))
(defmethod impl/handler :graph-wasm/set-objects
[message transfer]
(let [objects (:objects message)]
(rx/create
(fn [subs]
(-> @graph-wasm-module
(p/then (fn [module]
(if module
(try
(doseq [shape (vals objects)]
(set-object module shape))
(h/call module "_generate_db")
(rx/push! subs {:status :ok :processed (count objects)})
(rx/end! subs)
(catch :default cause
(log/error :hint "Error in graph-wasm/set-objects" :cause cause)
(rx/error! subs cause)
(rx/end! subs)))
(do
(log/warn :hint "Graph WASM module not available")
(rx/push! subs {:status :error :message "Module not available"})
(rx/end! subs)))))
(p/catch (fn [cause]
(log/error :hint "Error loading graph-wasm module" :cause cause)
(rx/error! subs cause)
(rx/end! subs))))
nil))))
(defmethod impl/handler :graph-wasm/search-similar-shapes
[message transfer]
(let [shape-id (:shape-id message)]
(rx/create
(fn [subs]
(-> @graph-wasm-module
(p/then (fn [module]
(if module
(try
(let [buffer (uuid/get-u32 shape-id)
ptr-raw (h/call module "_search_similar_shapes"
(aget buffer 0)
(aget buffer 1)
(aget buffer 2)
(aget buffer 3))
;; Convert pointer to unsigned 32-bit (handle negative numbers from WASM)
;; Use unsigned right shift to convert signed to unsigned 32-bit
ptr (unsigned-bit-shift-right ptr-raw 0)
heapu8 (unchecked-get module "HEAPU8")
;; Read count (first 4 bytes, little-endian u32)
count (bit-or (aget heapu8 ptr)
(bit-shift-left (aget heapu8 (+ ptr 1)) 8)
(bit-shift-left (aget heapu8 (+ ptr 2)) 16)
(bit-shift-left (aget heapu8 (+ ptr 3)) 24))
;; Read UUIDs (16 bytes each, starting at offset 4)
similar-shapes (loop [offset (+ ptr 4)
remaining count
result []]
(if (zero? remaining)
result
(let [uuid-bytes (.slice heapu8 offset (+ offset 16))]
(recur (+ offset 16)
(dec remaining)
(conj result (uuid/from-bytes uuid-bytes))))))]
;; Free the buffer
(h/call module "_free_similar_shapes_buffer")
(rx/push! subs {:status :ok :similar-shapes similar-shapes})
(rx/end! subs))
(catch :default cause
(log/error :hint "Error in graph-wasm/search-similar-shapes" :cause cause)
(rx/error! subs cause)
(rx/end! subs)))
(do
(log/warn :hint "Graph WASM module not available")
(rx/push! subs {:status :error :message "Module not available"})
(rx/end! subs)))))
(p/catch (fn [cause]
(log/error :hint "Error loading graph-wasm module" :cause cause)
(rx/error! subs cause)
(rx/end! subs))))
nil))))

View File

@@ -42,6 +42,37 @@
(deftest skips-when-no-svg-fill
(is (nil? (svg-fills/svg-fill->fills {:svg-attrs {:fill "none"}}))))
(def elliptical-shape
{:selrect {:x 0 :y 0 :width 200 :height 100}
:svg-attrs {:style {:fill "url(#grad-ellipse)"}}
:svg-defs {"grad-ellipse"
{:tag :radialGradient
:attrs {:id "grad-ellipse"
:gradientUnits "userSpaceOnUse"
:cx "50"
:cy "50"
:r "50"
:gradientTransform "matrix(2 0 0 1 0 0)"}
:content [{:tag :stop
:attrs {:offset "0"
:style "stop-color:#000000;stop-opacity:1"}}
{:tag :stop
:attrs {:offset "1"
:style "stop-color:#ffffff;stop-opacity:1"}}]}}})
(deftest builds-elliptical-radial-gradient-with-transform
(let [fills (svg-fills/svg-fill->fills elliptical-shape)
gradient (get-in (first fills) [:fill-color-gradient])]
(testing "ellipse from gradientTransform is preserved"
(is (= 1 (count fills)))
(is (= :radial (:type gradient)))
(is (= 0.5 (:start-x gradient)))
(is (= 0.5 (:start-y gradient)))
(is (= 0.5 (:end-x gradient)))
(is (= 1.0 (:end-y gradient)))
;; Scaling the X axis in the gradientTransform should reflect on width.
(is (= 1.0 (:width gradient))))))
(deftest resolve-shape-fills-prefers-existing-fills
(let [fills [{:fill-color "#ff00ff" :fill-opacity 0.75}]
resolved (svg-fills/resolve-shape-fills {:fills fills})]

View File

@@ -0,0 +1,52 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS INC
(ns frontend-tests.svg-filters-test
(:require
[app.render-wasm.svg-filters :as svg-filters]
[cljs.test :refer [deftest is testing]]))
(def sample-filter-shape
{:svg-attrs {:filter "url(#simple-filter)"}
:svg-defs {"simple-filter"
{:tag :filter
:content [{:tag :feOffset :attrs {:dx "2" :dy "3"}}
{:tag :feGaussianBlur :attrs {:stdDeviation "4"}}]}}})
(deftest derives-blur-and-shadow-from-svg-filter
(let [shape (svg-filters/apply-svg-filters sample-filter-shape)
blur (:blur shape)
shadow (:shadow shape)]
(testing "layer blur derived from feGaussianBlur"
(is (= :layer-blur (:type blur)))
(is (= 4.0 (:value blur))))
(testing "drop shadow derived from filter chain"
(is (= [{:style :drop-shadow
:offset-x 2.0
:offset-y 3.0
:blur 8.0
:spread 0
:hidden false
:color {:color "#000000" :opacity 1}}]
(map #(dissoc % :id) shadow))))
(testing "svg attrs remain intact"
(is (= "url(#simple-filter)" (get-in shape [:svg-attrs :filter]))))))
(deftest keeps-existing-native-filters
(let [existing {:blur {:id :existing :type :layer-blur :value 1.0}
:shadow [{:id :shadow :style :drop-shadow}]}
shape (svg-filters/apply-svg-filters (merge sample-filter-shape existing))]
(is (= (:blur existing) (:blur shape)))
(is (= (:shadow existing) (:shadow shape)))))
(deftest skips-when-no-filter-definition
(let [shape {:svg-attrs {:fill "#fff"}}
result (svg-filters/apply-svg-filters shape)]
(is (= shape result))))

View File

@@ -15,7 +15,7 @@
*/
export function addEventListeners(target, object, options) {
Object.entries(object).forEach(([type, listener]) =>
target.addEventListener(type, listener, options)
target.addEventListener(type, listener, options),
);
}
@@ -27,6 +27,6 @@ export function addEventListeners(target, object, options) {
*/
export function removeEventListeners(target, object) {
Object.entries(object).forEach(([type, listener]) =>
target.removeEventListener(type, listener)
target.removeEventListener(type, listener),
);
}

View File

@@ -664,8 +664,16 @@ export class TextEditor extends EventTarget {
* @param {boolean} allowHTMLPaste
* @returns {Root}
*/
export function createRootFromHTML(html, style = undefined, allowHTMLPaste = undefined) {
const fragment = mapContentFragmentFromHTML(html, style || undefined, allowHTMLPaste || undefined);
export function createRootFromHTML(
html,
style = undefined,
allowHTMLPaste = undefined,
) {
const fragment = mapContentFragmentFromHTML(
html,
style || undefined,
allowHTMLPaste || undefined,
);
const root = createRoot([], style);
root.replaceChildren(fragment);
resetInertElement();

View File

@@ -18,7 +18,10 @@ import { TextEditor } from "../TextEditor.js";
* @param {DataTransfer} clipboardData
* @returns {DocumentFragment}
*/
function getFormattedFragmentFromClipboardData(selectionController, clipboardData) {
function getFormattedFragmentFromClipboardData(
selectionController,
clipboardData,
) {
return mapContentFragmentFromHTML(
clipboardData.getData("text/html"),
selectionController.currentStyle,
@@ -79,9 +82,14 @@ export function paste(event, editor, selectionController) {
let fragment = null;
if (editor?.options?.allowHTMLPaste) {
fragment = getFormattedOrPlainFragmentFromClipboardData(event.clipboardData);
fragment = getFormattedOrPlainFragmentFromClipboardData(
event.clipboardData,
);
} else {
fragment = getPlainFragmentFromClipboardData(selectionController, event.clipboardData);
fragment = getPlainFragmentFromClipboardData(
selectionController,
event.clipboardData,
);
}
if (!fragment) {
@@ -92,10 +100,9 @@ export function paste(event, editor, selectionController) {
if (selectionController.isCollapsed) {
const hasOnlyOneParagraph = fragment.children.length === 1;
const hasOnlyOneTextSpan = fragment.firstElementChild.children.length === 1;
const forceTextSpan = fragment.firstElementChild.dataset.textSpan === "force";
if (hasOnlyOneParagraph
&& hasOnlyOneTextSpan
&& forceTextSpan) {
const forceTextSpan =
fragment.firstElementChild.dataset.textSpan === "force";
if (hasOnlyOneParagraph && hasOnlyOneTextSpan && forceTextSpan) {
selectionController.insertIntoFocus(fragment.textContent);
} else {
selectionController.insertPaste(fragment);
@@ -103,10 +110,9 @@ export function paste(event, editor, selectionController) {
} else {
const hasOnlyOneParagraph = fragment.children.length === 1;
const hasOnlyOneTextSpan = fragment.firstElementChild.children.length === 1;
const forceTextSpan = fragment.firstElementChild.dataset.textSpan === "force";
if (hasOnlyOneParagraph
&& hasOnlyOneTextSpan
&& forceTextSpan) {
const forceTextSpan =
fragment.firstElementChild.dataset.textSpan === "force";
if (hasOnlyOneParagraph && hasOnlyOneTextSpan && forceTextSpan) {
selectionController.replaceText(fragment.textContent);
} else {
selectionController.replaceWithPaste(fragment);

View File

@@ -23,7 +23,7 @@ export function deleteContentBackward(event, editor, selectionController) {
// If not is collapsed AKA is a selection, then
// we removeSelected.
if (!selectionController.isCollapsed) {
return selectionController.removeSelected({ direction: 'backward' });
return selectionController.removeSelected({ direction: "backward" });
}
// If we're in a text node and the offset is
@@ -32,18 +32,18 @@ export function deleteContentBackward(event, editor, selectionController) {
if (selectionController.isTextFocus && selectionController.focusOffset > 0) {
return selectionController.removeBackwardText();
// If we're in a text node but we're at the end of the
// paragraph, we should merge the current paragraph
// with the following paragraph.
// If we're in a text node but we're at the end of the
// paragraph, we should merge the current paragraph
// with the following paragraph.
} else if (
selectionController.isTextFocus &&
selectionController.focusAtStart
) {
return selectionController.mergeBackwardParagraph();
// If we're at an text span or a line break paragraph
// and there's more than one paragraph, then we should
// remove the next paragraph.
// If we're at an text span or a line break paragraph
// and there's more than one paragraph, then we should
// remove the next paragraph.
} else if (
selectionController.isTextSpanFocus ||
selectionController.isLineBreakFocus

View File

@@ -28,22 +28,21 @@ export function deleteContentForward(event, editor, selectionController) {
// If we're in a text node and the offset is
// greater than 0 (not at the start of the text span)
// we simple remove a character from the text.
if (selectionController.isTextFocus
&& selectionController.focusAtEnd) {
if (selectionController.isTextFocus && selectionController.focusAtEnd) {
return selectionController.mergeForwardParagraph();
// If we're in a text node but we're at the end of the
// paragraph, we should merge the current paragraph
// with the following paragraph.
// If we're in a text node but we're at the end of the
// paragraph, we should merge the current paragraph
// with the following paragraph.
} else if (
selectionController.isTextFocus &&
selectionController.focusOffset >= 0
) {
return selectionController.removeForwardText();
// If we're at a text span or a line break paragraph
// and there's more than one paragraph, then we should
// remove the next paragraph.
// If we're at a text span or a line break paragraph
// and there's more than one paragraph, then we should
// remove the next paragraph.
} else if (
(selectionController.isTextSpanFocus ||
selectionController.isLineBreakFocus) &&

View File

@@ -1,11 +1,17 @@
import { describe, test, expect } from 'vitest'
import { insertInto, removeBackward, removeForward, replaceWith } from './Text';
import { describe, test, expect } from "vitest";
import { insertInto, removeBackward, removeForward, replaceWith } from "./Text";
describe("Text", () => {
test("* should throw when passed wrong parameters", () => {
expect(() => insertInto(Infinity, Infinity, Infinity)).toThrowError('Invalid string');
expect(() => insertInto('Hello', Infinity, Infinity)).toThrowError('Invalid offset');
expect(() => insertInto('Hello', 0, Infinity)).toThrowError('Invalid string');
expect(() => insertInto(Infinity, Infinity, Infinity)).toThrowError(
"Invalid string",
);
expect(() => insertInto("Hello", Infinity, Infinity)).toThrowError(
"Invalid offset",
);
expect(() => insertInto("Hello", 0, Infinity)).toThrowError(
"Invalid string",
);
});
test("`insertInto` should insert a string into an offset", () => {
@@ -13,7 +19,9 @@ describe("Text", () => {
});
test("`replaceWith` should replace a string into a string", () => {
expect(replaceWith("Hello, Something!", 7, 16, "World")).toBe("Hello, World!");
expect(replaceWith("Hello, Something!", 7, 16, "World")).toBe(
"Hello, World!",
);
});
test("`removeBackward` should remove string backward from start (offset 0)", () => {
@@ -26,13 +34,13 @@ describe("Text", () => {
test("`removeBackward` should remove string backward from end", () => {
expect(removeBackward("Hello, World!", "Hello, World!".length)).toBe(
"Hello, World"
"Hello, World",
);
});
test("`removeForward` should remove string forward from end", () => {
expect(removeForward("Hello, World!", "Hello, World!".length)).toBe(
"Hello, World!"
"Hello, World!",
);
});

View File

@@ -24,7 +24,7 @@ function getContext() {
if (!context) {
context = canvas.getContext("2d");
}
return context
return context;
}
/**

View File

@@ -230,15 +230,10 @@ export function mapContentFragmentFromString(string, styleDefaults) {
const fragment = document.createDocumentFragment();
for (const line of lines) {
if (line === "") {
fragment.appendChild(
createEmptyParagraph(styleDefaults)
);
fragment.appendChild(createEmptyParagraph(styleDefaults));
} else {
const textSpan = createTextSpan(new Text(line), styleDefaults);
const paragraph = createParagraph(
[textSpan],
styleDefaults,
);
const paragraph = createParagraph([textSpan], styleDefaults);
if (lines.length === 1) {
paragraph.dataset.textSpan = "force";
}

View File

@@ -112,7 +112,11 @@ describe("Paragraph", () => {
const helloTextSpan = createTextSpan(new Text("Hello, "));
const worldTextSpan = createTextSpan(new Text("World"));
const exclTextSpan = createTextSpan(new Text("!"));
const paragraph = createParagraph([helloTextSpan, worldTextSpan, exclTextSpan]);
const paragraph = createParagraph([
helloTextSpan,
worldTextSpan,
exclTextSpan,
]);
const newParagraph = splitParagraphAtNode(paragraph, 1);
expect(newParagraph).toBeInstanceOf(HTMLDivElement);
expect(newParagraph.nodeName).toBe(TAG);

View File

@@ -1,5 +1,11 @@
import { describe, test, expect } from "vitest";
import { createEmptyRoot, createRoot, setRootStyles, TAG, TYPE } from "./Root.js";
import {
createEmptyRoot,
createRoot,
setRootStyles,
TAG,
TYPE,
} from "./Root.js";
/* @vitest-environment jsdom */
describe("Root", () => {

View File

@@ -6,7 +6,7 @@
* Copyright (c) KALEIDOS INC
*/
import StyleDeclaration from '../../controllers/StyleDeclaration.js';
import StyleDeclaration from "../../controllers/StyleDeclaration.js";
import { getFills } from "./Color.js";
const DEFAULT_FONT_SIZE = "16px";
@@ -339,8 +339,7 @@ export function setStylesFromObject(element, allowedStyles, styleObject) {
continue;
}
let styleValue = styleObject[styleName];
if (!styleValue)
continue;
if (!styleValue) continue;
if (styleName === "font-family") {
styleValue = sanitizeFontFamily(styleValue);
@@ -388,8 +387,10 @@ export function setStylesFromDeclaration(
* @returns {HTMLElement}
*/
export function setStyles(element, allowedStyles, styleObjectOrDeclaration) {
if (styleObjectOrDeclaration instanceof CSSStyleDeclaration
|| styleObjectOrDeclaration instanceof StyleDeclaration) {
if (
styleObjectOrDeclaration instanceof CSSStyleDeclaration ||
styleObjectOrDeclaration instanceof StyleDeclaration
) {
return setStylesFromDeclaration(
element,
allowedStyles,

View File

@@ -22,8 +22,7 @@ import { isRoot } from "./Root.js";
*/
export function isTextNode(node) {
if (!node) throw new TypeError("Invalid text node");
return node.nodeType === Node.TEXT_NODE
|| isLineBreak(node);
return node.nodeType === Node.TEXT_NODE || isLineBreak(node);
}
/**
@@ -33,8 +32,7 @@ export function isTextNode(node) {
* @returns {boolean}
*/
export function isEmptyTextNode(node) {
return node.nodeType === Node.TEXT_NODE
&& node.nodeValue === "";
return node.nodeType === Node.TEXT_NODE && node.nodeValue === "";
}
/**

View File

@@ -6,7 +6,7 @@
* Copyright (c) KALEIDOS INC
*/
import SafeGuard from '../../controllers/SafeGuard.js';
import SafeGuard from "../../controllers/SafeGuard.js";
/**
* Iterator direction.
@@ -58,7 +58,7 @@ export class TextNodeIterator {
startNode,
rootNode,
skipNodes = new Set(),
direction = TextNodeIteratorDirection.FORWARD
direction = TextNodeIteratorDirection.FORWARD,
) {
if (startNode === rootNode) {
return TextNodeIterator.findDown(
@@ -67,7 +67,7 @@ export class TextNodeIterator {
: startNode.lastChild,
rootNode,
skipNodes,
direction
direction,
);
}
@@ -95,7 +95,7 @@ export class TextNodeIterator {
: currentNode.lastChild,
rootNode,
skipNodes,
direction
direction,
);
}
currentNode =
@@ -119,7 +119,7 @@ export class TextNodeIterator {
startNode,
rootNode,
backTrack = new Set(),
direction = TextNodeIteratorDirection.FORWARD
direction = TextNodeIteratorDirection.FORWARD,
) {
backTrack.add(startNode);
if (TextNodeIterator.isTextNode(startNode)) {
@@ -127,14 +127,14 @@ export class TextNodeIterator {
startNode.parentNode,
rootNode,
backTrack,
direction
direction,
);
} else if (TextNodeIterator.isContainerNode(startNode)) {
const found = TextNodeIterator.findDown(
startNode,
rootNode,
backTrack,
direction
direction,
);
if (found) {
return found;
@@ -144,7 +144,7 @@ export class TextNodeIterator {
startNode.parentNode,
rootNode,
backTrack,
direction
direction,
);
}
}
@@ -214,7 +214,7 @@ export class TextNodeIterator {
this.#currentNode,
this.#rootNode,
new Set(),
TextNodeIteratorDirection.FORWARD
TextNodeIteratorDirection.FORWARD,
);
if (!nextNode) {
@@ -237,7 +237,7 @@ export class TextNodeIterator {
this.#currentNode,
this.#rootNode,
new Set(),
TextNodeIteratorDirection.BACKWARD
TextNodeIteratorDirection.BACKWARD,
);
if (!previousNode) {
@@ -270,10 +270,8 @@ export class TextNodeIterator {
* @param {TextNode} endNode
* @yields {TextNode}
*/
* iterateFrom(startNode, endNode) {
const comparedPosition = startNode.compareDocumentPosition(
endNode
);
*iterateFrom(startNode, endNode) {
const comparedPosition = startNode.compareDocumentPosition(endNode);
this.#currentNode = startNode;
SafeGuard.start();
while (this.#currentNode !== endNode) {

View File

@@ -38,7 +38,7 @@ export class ChangeController extends EventTarget {
* @param {number} [time=500]
*/
constructor(time = 500) {
super()
super();
if (typeof time === "number" && (!Number.isInteger(time) || time <= 0)) {
throw new TypeError("Invalid time");
}

View File

@@ -24,19 +24,19 @@ export function start() {
*/
export function update() {
if (Date.now - startTime >= SAFE_GUARD_TIME) {
throw new Error('Safe guard timeout');
throw new Error("Safe guard timeout");
}
}
let timeoutId = 0
let timeoutId = 0;
export function throwAfter(error, timeout = SAFE_GUARD_TIME) {
timeoutId = setTimeout(() => {
throw error
}, timeout)
throw error;
}, timeout);
}
export function throwCancel() {
clearTimeout(timeoutId)
clearTimeout(timeoutId);
}
export default {

View File

@@ -54,7 +54,7 @@ import { isRoot, setRootStyles } from "../content/dom/Root.js";
import { SelectionDirection } from "./SelectionDirection.js";
import SafeGuard from "./SafeGuard.js";
import { sanitizeFontFamily } from "../content/dom/Style.js";
import StyleDeclaration from './StyleDeclaration.js';
import StyleDeclaration from "./StyleDeclaration.js";
/**
* Supported options for the SelectionController.
@@ -280,11 +280,17 @@ export class SelectionController extends EventTarget {
// FIXME: I don't like this approximation. Having to iterate nodes twice
// is bad for performance. I think we need another way of "computing"
// the cascade.
for (const textNode of this.#textNodeIterator.iterateFrom(startNode, endNode)) {
for (const textNode of this.#textNodeIterator.iterateFrom(
startNode,
endNode,
)) {
const paragraph = textNode.parentElement.parentElement;
this.#applyStylesFromElementToCurrentStyle(paragraph);
}
for (const textNode of this.#textNodeIterator.iterateFrom(startNode, endNode)) {
for (const textNode of this.#textNodeIterator.iterateFrom(
startNode,
endNode,
)) {
const textSpan = textNode.parentElement;
this.#mergeStylesFromElementToCurrentStyle(textSpan);
}
@@ -498,19 +504,12 @@ export class SelectionController extends EventTarget {
if (!this.#savedSelection) return false;
if (this.#savedSelection.anchorNode && this.#savedSelection.focusNode) {
if (this.#savedSelection.anchorNode === this.#savedSelection.focusNode) {
this.#selection.setPosition(
this.#savedSelection.focusNode,
this.#savedSelection.focusOffset,
);
} else {
this.#selection.setBaseAndExtent(
this.#savedSelection.anchorNode,
this.#savedSelection.anchorOffset,
this.#savedSelection.focusNode,
this.#savedSelection.focusOffset,
);
}
this.#selection.setBaseAndExtent(
this.#savedSelection.anchorNode,
this.#savedSelection.anchorOffset,
this.#savedSelection.focusNode,
this.#savedSelection.focusOffset,
);
}
this.#savedSelection = null;
return true;
@@ -1132,10 +1131,7 @@ export class SelectionController extends EventTarget {
const hasOnlyOneParagraph = fragment.children.length === 1;
const forceTextSpan =
fragment.firstElementChild?.dataset?.textSpan === "force";
if (
hasOnlyOneParagraph &&
forceTextSpan
) {
if (hasOnlyOneParagraph && forceTextSpan) {
// first text span
const collapseNode = fragment.firstElementChild.firstElementChild;
if (this.isTextSpanStart) {
@@ -1403,7 +1399,7 @@ export class SelectionController extends EventTarget {
// the focus node is a <span>.
if (isTextSpan(this.focusNode)) {
this.focusNode.firstElementChild.replaceWith(textNode);
// the focus node is a <br>.
// the focus node is a <br>.
} else {
this.focusNode.replaceWith(textNode);
}
@@ -1981,8 +1977,7 @@ export class SelectionController extends EventTarget {
this.setSelection(newTextSpan.firstChild, 0, newTextSpan.firstChild, 0);
}
// The styles are applied to the paragraph
else
{
else {
const paragraph = this.startParagraph;
setParagraphStyles(paragraph, newStyles);
// Apply styles to child text spans.

View File

@@ -278,9 +278,9 @@ describe("SelectionController", () => {
expect(textEditorMock.root.firstChild.firstChild.firstChild.nodeValue).toBe(
"Hello",
);
expect(
textEditorMock.root.lastChild.firstChild.firstChild.nodeValue,
).toBe(", World!");
expect(textEditorMock.root.lastChild.firstChild.firstChild.nodeValue).toBe(
", World!",
);
});
test("`insertPaste` should insert a paragraph from a pasted fragment (at middle)", () => {
@@ -292,7 +292,12 @@ describe("SelectionController", () => {
textEditorMock,
selection,
);
focus(selection, textEditorMock, root.firstChild.firstChild.firstChild, "Lorem ".length);
focus(
selection,
textEditorMock,
root.firstChild.firstChild.firstChild,
"Lorem ".length,
);
const paragraph = createParagraph([createTextSpan(new Text("ipsum "))]);
const fragment = document.createDocumentFragment();
fragment.append(paragraph);
@@ -315,9 +320,9 @@ describe("SelectionController", () => {
expect(textEditorMock.root.firstChild.firstChild.firstChild.nodeValue).toBe(
"Lorem ",
);
expect(textEditorMock.root.children.item(1).firstChild.firstChild.nodeValue).toBe(
"ipsum ",
);
expect(
textEditorMock.root.children.item(1).firstChild.firstChild.nodeValue,
).toBe("ipsum ");
expect(textEditorMock.root.lastChild.firstChild.firstChild.nodeValue).toBe(
"dolor",
);
@@ -359,25 +364,21 @@ describe("SelectionController", () => {
expect(textEditorMock.root.firstChild.firstChild.firstChild.nodeValue).toBe(
"Hello",
);
expect(
textEditorMock.root.lastChild.firstChild.firstChild.nodeValue,
).toBe(", World!");
expect(textEditorMock.root.lastChild.firstChild.firstChild.nodeValue).toBe(
", World!",
);
});
test("`insertPaste` should insert a text span from a pasted fragment (at start)", () => {
const textEditorMock = TextEditorMock.createTextEditorMockWithText(", World!");
const textEditorMock =
TextEditorMock.createTextEditorMockWithText(", World!");
const root = textEditorMock.root;
const selection = document.getSelection();
const selectionController = new SelectionController(
textEditorMock,
selection,
);
focus(
selection,
textEditorMock,
root.firstChild.firstChild.firstChild,
0,
);
focus(selection, textEditorMock, root.firstChild.firstChild.firstChild, 0);
const paragraph = createParagraph([createTextSpan(new Text("Hello"))]);
paragraph.dataset.textSpan = "force";
const fragment = document.createDocumentFragment();
@@ -415,7 +416,12 @@ describe("SelectionController", () => {
textEditorMock,
selection,
);
focus(selection, textEditorMock, root.firstChild.firstChild.firstChild, "Lorem ".length);
focus(
selection,
textEditorMock,
root.firstChild.firstChild.firstChild,
"Lorem ".length,
);
const paragraph = createParagraph([createTextSpan(new Text("ipsum "))]);
paragraph.dataset.textSpan = "force";
const fragment = document.createDocumentFragment();
@@ -439,9 +445,9 @@ describe("SelectionController", () => {
expect(textEditorMock.root.firstChild.firstChild.firstChild.nodeValue).toBe(
"Lorem ",
);
expect(textEditorMock.root.firstChild.children.item(1).firstChild.nodeValue).toBe(
"ipsum ",
);
expect(
textEditorMock.root.firstChild.children.item(1).firstChild.nodeValue,
).toBe("ipsum ");
expect(
textEditorMock.root.firstChild.children.item(2).firstChild.nodeValue,
).toBe("dolor");
@@ -461,9 +467,7 @@ describe("SelectionController", () => {
root.firstChild.firstChild.firstChild,
"Hello".length,
);
const paragraph = createParagraph([
createTextSpan(new Text(", World!"))
]);
const paragraph = createParagraph([createTextSpan(new Text(", World!"))]);
paragraph.dataset.textSpan = "force";
const fragment = document.createDocumentFragment();
fragment.append(paragraph);
@@ -486,9 +490,9 @@ describe("SelectionController", () => {
expect(textEditorMock.root.firstChild.firstChild.firstChild.nodeValue).toBe(
"Hello",
);
expect(textEditorMock.root.firstChild.children.item(1).firstChild.nodeValue).toBe(
", World!",
);
expect(
textEditorMock.root.firstChild.children.item(1).firstChild.nodeValue,
).toBe(", World!");
});
test("`removeBackwardText` should remove text in backward direction (backspace)", () => {

View File

@@ -77,7 +77,10 @@ export class StyleDeclaration {
const currentValue = this.getPropertyValue(name);
if (this.#isQuotedValue(currentValue, value)) {
return this.setProperty(name, value);
} else if (currentValue === "" && value === StyleDeclaration.Property.NULL) {
} else if (
currentValue === "" &&
value === StyleDeclaration.Property.NULL
) {
return this.setProperty(name, value);
} else if (currentValue === "" && ["initial", "none"].includes(value)) {
return this.setProperty(name, value);
@@ -107,4 +110,4 @@ export class StyleDeclaration {
}
}
export default StyleDeclaration
export default StyleDeclaration;

View File

@@ -43,33 +43,38 @@ export class SelectionControllerDebug {
this.#elements.isParagraphStart.checked =
selectionController.isParagraphStart;
this.#elements.isParagraphEnd.checked = selectionController.isParagraphEnd;
this.#elements.isTextSpanStart.checked = selectionController.isTextSpanStart;
this.#elements.isTextSpanStart.checked =
selectionController.isTextSpanStart;
this.#elements.isTextSpanEnd.checked = selectionController.isTextSpanEnd;
this.#elements.isTextAnchor.checked = selectionController.isTextAnchor;
this.#elements.isTextFocus.checked = selectionController.isTextFocus;
this.#elements.focusNode.value = this.getNodeDescription(
selectionController.focusNode,
selectionController.focusOffset
selectionController.focusOffset,
);
this.#elements.focusOffset.value = selectionController.focusOffset;
this.#elements.anchorNode.value = this.getNodeDescription(
selectionController.anchorNode,
selectionController.anchorOffset
selectionController.anchorOffset,
);
this.#elements.anchorOffset.value = selectionController.anchorOffset;
this.#elements.focusTextSpan.value = this.getNodeDescription(
selectionController.focusTextSpan
selectionController.focusTextSpan,
);
this.#elements.anchorTextSpan.value = this.getNodeDescription(
selectionController.anchorTextSpan
selectionController.anchorTextSpan,
);
this.#elements.focusParagraph.value = this.getNodeDescription(
selectionController.focusParagraph
selectionController.focusParagraph,
);
this.#elements.anchorParagraph.value = this.getNodeDescription(
selectionController.anchorParagraph
selectionController.anchorParagraph,
);
this.#elements.startContainer.value = this.getNodeDescription(
selectionController.startContainer,
);
this.#elements.endContainer.value = this.getNodeDescription(
selectionController.endContainer,
);
this.#elements.startContainer.value = this.getNodeDescription(selectionController.startContainer);
this.#elements.endContainer.value = this.getNodeDescription(selectionController.endContainer);
}
}

View File

@@ -39,10 +39,7 @@ export class Point {
}
polar(angle, length = 1.0) {
return this.set(
Math.cos(angle) * length,
Math.sin(angle) * length
);
return this.set(Math.cos(angle) * length, Math.sin(angle) * length);
}
add({ x, y }) {
@@ -119,10 +116,7 @@ export class Point {
export class Rect {
static create(x, y, width, height) {
return new Rect(
new Point(width, height),
new Point(x, y),
);
return new Rect(new Point(width, height), new Point(x, y));
}
#size;
@@ -228,10 +222,7 @@ export class Rect {
}
clone() {
return new Rect(
this.#size.clone(),
this.#position.clone(),
);
return new Rect(this.#size.clone(), this.#position.clone());
}
toFixed(fractionDigits = 0) {

View File

@@ -82,13 +82,13 @@ export class Shape {
}
get rotation() {
return this.#rotation
return this.#rotation;
}
set rotation(newRotation) {
if (!Number.isFinite(newRotation)) {
throw new TypeError('Invalid rotation')
throw new TypeError("Invalid rotation");
}
this.#rotation = newRotation
this.#rotation = newRotation;
}
}

View File

@@ -6,8 +6,7 @@ export function fromStyle(style) {
const entry = Object.entries(this).find(([name, value]) =>
name === fromStyleValue(style) ? value : 0,
);
if (!entry)
return;
if (!entry) return;
const [name] = entry;
return name;

View File

@@ -1,4 +1,4 @@
import { Point } from './geom';
import { Point } from "./geom";
export class Viewport {
#zoom;
@@ -38,7 +38,7 @@ export class Viewport {
}
pan(dx, dy) {
this.#position.x += dx / this.#zoom
this.#position.y += dy / this.#zoom
this.#position.x += dx / this.#zoom;
this.#position.y += dy / this.#zoom;
}
}

View File

@@ -1,6 +1,9 @@
import { createRoot } from "../editor/content/dom/Root.js";
import { createParagraph } from "../editor/content/dom/Paragraph.js";
import { createEmptyTextSpan, createTextSpan } from "../editor/content/dom/TextSpan.js";
import {
createEmptyTextSpan,
createTextSpan,
} from "../editor/content/dom/TextSpan.js";
import { createLineBreak } from "../editor/content/dom/LineBreak.js";
export class TextEditorMock extends EventTarget {
@@ -38,14 +41,14 @@ export class TextEditorMock extends EventTarget {
static createTextEditorMockWithRoot(root) {
const container = TextEditorMock.getTemplate();
const selectionImposterElement = container.querySelector(
".text-editor-selection-imposter"
".text-editor-selection-imposter",
);
const textEditorMock = new TextEditorMock(
container.querySelector(".text-editor-content"),
{
root,
selectionImposterElement,
}
},
);
return textEditorMock;
}
@@ -86,8 +89,8 @@ export class TextEditorMock extends EventTarget {
return this.createTextEditorMockWithParagraphs([
createParagraph([
text.length === 0
? createEmptyTextSpan()
: createTextSpan(new Text(text))
? createEmptyTextSpan()
: createTextSpan(new Text(text)),
]),
]);
}
@@ -100,7 +103,9 @@ export class TextEditorMock extends EventTarget {
* @returns
*/
static createTextEditorMockWithParagraph(textSpans) {
return this.createTextEditorMockWithParagraphs([createParagraph(textSpans)]);
return this.createTextEditorMockWithParagraphs([
createParagraph(textSpans),
]);
}
#element = null;

View File

@@ -1,30 +1,28 @@
import path from "node:path";
import fs from 'node:fs/promises';
import fs from "node:fs/promises";
import { defineConfig } from "vite";
import { coverageConfigDefaults } from "vitest/config";
async function waitFor(timeInMillis) {
return new Promise(resolve =>
setTimeout(_ => resolve(), timeInMillis)
);
return new Promise((resolve) => setTimeout((_) => resolve(), timeInMillis));
}
const wasmWatcherPlugin = (options = {}) => {
return {
name: "vite-wasm-watcher-plugin",
configureServer(server) {
server.watcher.add("../resources/public/js/render_wasm.wasm")
server.watcher.add("../resources/public/js/render_wasm.js")
server.watcher.add("../resources/public/js/render_wasm.wasm");
server.watcher.add("../resources/public/js/render_wasm.js");
server.watcher.on("change", async (file) => {
if (file.includes("../resources/")) {
// If we copy the files immediately, we end
// up with an empty .js file (I don't know why).
await waitFor(100)
await waitFor(100);
// copy files.
await fs.copyFile(
path.resolve(file),
path.resolve('./src/wasm/', path.basename(file))
)
path.resolve("./src/wasm/", path.basename(file)),
);
console.log(`${file} changed`);
}
});
@@ -49,9 +47,7 @@ const wasmWatcherPlugin = (options = {}) => {
};
export default defineConfig({
plugins: [
wasmWatcherPlugin()
],
plugins: [wasmWatcherPlugin()],
root: "./src",
resolve: {
alias: {

View File

@@ -0,0 +1,6 @@
[target.wasm32-unknown-emscripten]
# Note: Not using atomics to avoid recompiling std library
# We're running without pthreads, so lbug needs to work single-threaded
rustflags = ["-C", "link-arg=-fexceptions"]
# Linker is configured via environment variable CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_LINKER in _build_env

5
graph-wasm/.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
target/
debug/
**/*.rs.bk

486
graph-wasm/Cargo.lock generated Normal file
View File

@@ -0,0 +1,486 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "anstyle"
version = "1.0.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5192cca8006f1fd4f7237516f40fa183bb07f8fbdfedaa0036de5ea9b0b45e78"
[[package]]
name = "arrayvec"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
[[package]]
name = "autocfg"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "bumpalo"
version = "3.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46c5e41b57b8bba42a04676d81cb89e9ee8e859a1a66f80a5a72e1cb76b34d43"
[[package]]
name = "cc"
version = "1.2.49"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90583009037521a116abf44494efecd645ba48b6622457080f080b85544e2215"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "clap"
version = "4.5.53"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c9e340e012a1bf4935f5282ed1436d1489548e8f72308207ea5df0e23d2d03f8"
dependencies = [
"clap_builder",
]
[[package]]
name = "clap_builder"
version = "4.5.53"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d76b5d13eaa18c901fd2f7fca939fefe3a0727a953561fefdf3b2922b8569d00"
dependencies = [
"anstyle",
"clap_lex",
"strsim",
]
[[package]]
name = "clap_lex"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1d728cc89cf3aee9ff92b05e62b19ee65a02b5702cff7d5a377e32c6ae29d8d"
[[package]]
name = "cmake"
version = "0.1.56"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b042e5d8a74ae91bb0961acd039822472ec99f8ab0948cbf6d1369588f8be586"
dependencies = [
"cc",
]
[[package]]
name = "codespan-reporting"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3538270d33cc669650c4b093848450d380def10c331d38c768e34cac80576e6e"
dependencies = [
"termcolor",
"unicode-width",
]
[[package]]
name = "cxx"
version = "1.0.138"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3956d60afa98653c5a57f60d7056edd513bfe0307ef6fb06f6167400c3884459"
dependencies = [
"cc",
"cxxbridge-cmd",
"cxxbridge-flags",
"cxxbridge-macro",
"foldhash",
"link-cplusplus",
]
[[package]]
name = "cxx-build"
version = "1.0.138"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a4b7522f539fe056f1d6fc8577d8ab731451f6f33a89b1e5912e22b76c553e7"
dependencies = [
"cc",
"codespan-reporting",
"proc-macro2",
"quote",
"scratch",
"syn",
]
[[package]]
name = "cxxbridge-cmd"
version = "1.0.138"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f01e92ab4ce9fd4d16e3bb11b158d98cbdcca803c1417aa43130a6526fbf208"
dependencies = [
"clap",
"codespan-reporting",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "cxxbridge-flags"
version = "1.0.138"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c41cbfab344869e70998b388923f7d1266588f56c8ca284abf259b1c1ffc695"
[[package]]
name = "cxxbridge-macro"
version = "1.0.138"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88d82a2f759f0ad3eae43b96604efd42b1d4729a35a6f2dc7bdb797ae25d9284"
dependencies = [
"proc-macro2",
"quote",
"rustversion",
"syn",
]
[[package]]
name = "deranged"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ececcb659e7ba858fb4f10388c250a7252eb0a27373f1a72b8748afdd248e587"
dependencies = [
"powerfmt",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a3076410a55c90011c298b04d0cfa770b00fa04e1e3c97d3f6c9de105a03844"
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "getrandom"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"wasip2",
]
[[package]]
name = "graph"
version = "0.1.0"
dependencies = [
"lbug",
"uuid",
]
[[package]]
name = "js-sys"
version = "0.3.83"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "464a3709c7f55f1f721e5389aa6ea4e3bc6aba669353300af094b29ffbdde1d8"
dependencies = [
"once_cell",
"wasm-bindgen",
]
[[package]]
name = "lbug"
version = "0.12.2"
dependencies = [
"cmake",
"cxx",
"cxx-build",
"rust_decimal",
"rustversion",
"time",
"uuid",
]
[[package]]
name = "libc"
version = "0.2.178"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37c93d8daa9d8a012fd8ab92f088405fb202ea0b6ab73ee2482ae66af4f42091"
[[package]]
name = "link-cplusplus"
version = "1.0.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f78c730aaa7d0b9336a299029ea49f9ee53b0ed06e9202e8cb7db9bae7b8c82"
dependencies = [
"cc",
]
[[package]]
name = "num-conv"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51d515d32fb182ee37cda2ccdcb92950d6a3c2893aa280e540671c2cd0f3b1d9"
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "powerfmt"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
[[package]]
name = "proc-macro2"
version = "1.0.103"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ee95bc4ef87b8d5ba32e8b7714ccc834865276eab0aed5c9958d00ec45f49e8"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a338cc41d27e6cc6dce6cefc13a0729dfbb81c262b1f519331575dd80ef3067f"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "rust_decimal"
version = "1.39.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35affe401787a9bd846712274d97654355d21b2a2c092a3139aabe31e9022282"
dependencies = [
"arrayvec",
"num-traits",
]
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "scratch"
version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d68f2ec51b097e4c1a75b681a8bec621909b5e91f15bb7b840c4f2f7b01148b2"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "shlex"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
[[package]]
name = "strsim"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "syn"
version = "2.0.111"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "390cc9a294ab71bdb1aa2e99d13be9c753cd2d7bd6560c77118597410c4d2e87"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "termcolor"
version = "1.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06794f8f6c5c898b3275aebefa6b8a1cb24cd2c6c79397ab15774837a0bc5755"
dependencies = [
"winapi-util",
]
[[package]]
name = "time"
version = "0.3.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91e7d9e3bb61134e77bde20dd4825b97c010155709965fedf0f49bb138e52a9d"
dependencies = [
"deranged",
"num-conv",
"powerfmt",
"serde",
"time-core",
]
[[package]]
name = "time-core"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40868e7c1d2f0b8d73e4a8c7f0ff63af4f6d19be117e90bd73eb1d62cf831c6b"
[[package]]
name = "unicode-ident"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9312f7c4f6ff9069b165498234ce8be658059c6728633667c526e27dc2cf1df5"
[[package]]
name = "unicode-width"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af"
[[package]]
name = "uuid"
version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e2e054861b4bd027cd373e18e8d8d8e6548085000e41290d95ce0c373a654b4a"
dependencies = [
"getrandom",
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "wasip2"
version = "1.0.1+wasi-0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0562428422c63773dad2c345a1882263bbf4d65cf3f42e90921f787ef5ad58e7"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d759f433fa64a2d763d1340820e46e111a7a5ab75f993d1852d70b03dbb80fd"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48cb0d2638f8baedbc542ed444afc0644a29166f1595371af4fecf8ce1e7eeb3"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cefb59d5cd5f92d9dcf80e4683949f15ca4b511f4ac0a6e14d4e1ac60c6ecd40"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cbc538057e648b67f72a982e708d485b2efa771e1ac05fec311f9f63e5800db4"
dependencies = [
"unicode-ident",
]
[[package]]
name = "winapi-util"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "wit-bindgen"
version = "0.46.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f17a85883d4e6d00e8a97c586de764dabcc06133f7f1d55dce5cdc070ad7fe59"

31
graph-wasm/Cargo.toml Normal file
View File

@@ -0,0 +1,31 @@
[package]
name = "graph"
version = "0.1.0"
edition = "2021"
repository = "https://github.com/penpot/penpot"
license-file = "../LICENSE"
description = "Wasm-based graph module for Penpot"
build = "build.rs"
[[bin]]
name = "graph_wasm"
path = "src/main.rs"
[profile.release]
opt-level = "s"
# Note: We need panic=unwind because core requires it
# We'll compile std from source with atomics support instead
[profile.dev]
# Note: We need panic=unwind because core requires it
# We'll compile std from source with atomics support instead
[dependencies]
lbug = "0.12.2"
uuid = { version = "1.11.0", features = ["v4", "js"] }
# Patch lbug to use local version with wasm32-unknown-emscripten support
[patch.crates-io]
lbug = { path = "./lbug-0.12.2" }

197
graph-wasm/README_WASM.md Normal file
View File

@@ -0,0 +1,197 @@
# lbug WASM Support
This document describes the modifications made to the `lbug` crate to enable compilation for the `wasm32-unknown-emscripten` target, and the build requirements for using it in a WASM context.
## Overview
The `lbug` crate is a Rust wrapper around a C++ graph database library. To compile it for WebAssembly using Emscripten, several modifications were necessary to handle:
1. C++ exception handling in WASM
2. Conditional compilation for WASM-specific code paths
3. Proper linking of static libraries for Emscripten
4. CMake configuration for single-threaded mode
## Changes Made to lbug
### 1. `build.rs` Modifications
The build script (`build.rs`) was modified to detect and handle the `wasm32-unknown-emscripten` target:
#### WASM Detection
```rust
fn is_wasm_emscripten() -> bool {
env::var("TARGET")
.map(|t| t == "wasm32-unknown-emscripten")
.unwrap_or(false)
}
```
#### CMake Configuration (`build_bundled_cmake()`)
- **Single-threaded mode**: Sets `SINGLE_THREADED=TRUE` for WASM builds (required by Emscripten)
- The Emscripten toolchain is automatically detected when `CC`/`CXX` point to `emcc`/`em++`
#### FFI Build Configuration (`build_ffi()`)
- **C++20 standard**: Uses `-std=c++20` flag for WASM
- **Exception support**: Enables `-fexceptions` flag (exceptions must be enabled at compile time)
- Note: `-sDISABLE_EXCEPTION_CATCHING=0` is a linker flag and should be set via `EMCC_CFLAGS`
#### Library Linking (`link_libraries()`)
- **Explicit dependency linking**: For WASM, all static dependencies are explicitly linked:
- `utf8proc`, `antlr4_cypher`, `antlr4_runtime`, `re2`, `fastpfor`
- `parquet`, `thrift`, `snappy`, `zstd`, `miniz`
- `mbedtls`, `brotlidec`, `brotlicommon`, `lz4`
- `roaring_bitmap`, `simsimd`
- **Linking order**: Libraries are linked after FFI compilation for WASM (different from native builds)
### 2. `src/error.rs` Modifications
The error handling code was modified to conditionally compile C++ exception support:
#### Conditional C++ Exception Variant
The `Error::CxxException` variant and related implementations are conditionally compiled:
```rust
#[cfg(not(target_arch = "wasm32"))]
pub enum Error {
// ... other variants ...
CxxException(cxx::Exception),
// ...
}
```
#### Exception Mapping for WASM
In WASM builds, `cxx::Exception` is mapped to `Error::FailedQuery`:
```rust
impl From<cxx::Exception> for Error {
fn from(item: cxx::Exception) -> Self {
#[cfg(not(target_arch = "wasm32"))]
{
Error::CxxException(item)
}
#[cfg(target_arch = "wasm32")]
{
// In wasm, CxxException is not available, map to a generic error
Error::FailedQuery(item.to_string())
}
}
}
```
**Note**: This change does not affect the rest of `lbug` due to `#[cfg]` guards, ensuring native builds remain unchanged.
## Build Requirements
### 1. Using the Modified lbug Crate
To use the modified `lbug` crate in your project, add a `[patch.crates-io]` section to your `Cargo.toml`:
```toml
[dependencies]
lbug = "0.12.2"
# Patch lbug to use local version with wasm32-unknown-emscripten support
[patch.crates-io]
lbug = { path = "./lbug-0.12.2" }
```
### 2. Emscripten Environment Setup
The build requires Emscripten to be properly configured. The following environment variables should be set:
#### Memory Configuration
```bash
export EM_INITIAL_HEAP=$((256 * 1024 * 1024)) # 256 MB initial heap
export EM_MAXIMUM_MEMORY=$((4 * 1024 * 1024 * 1024)) # 4 GB maximum
export EM_MEMORY_GROWTH_GEOMETRIC_STEP=0.8
export EM_MALLOC=dlmalloc
```
#### Compiler/Linker Configuration
```bash
# Prevent cc-rs from adding default flags that conflict with Emscripten
export CRATE_CC_NO_DEFAULTS=1
# Emscripten compiler flags
export EMCC_CFLAGS="--no-entry \
-sASSERTIONS=1 \
-sALLOW_TABLE_GROWTH=1 \
-sALLOW_MEMORY_GROWTH=1 \
-sINITIAL_HEAP=$EM_INITIAL_HEAP \
-sMEMORY_GROWTH_GEOMETRIC_STEP=$EM_MEMORY_GROWTH_GEOMETRIC_STEP \
-sMAXIMUM_MEMORY=$EM_MAXIMUM_MEMORY \
-sERROR_ON_UNDEFINED_SYMBOLS=0 \
-sDISABLE_EXCEPTION_CATCHING=0 \
-sEXPORT_NAME=createGraphModule \
-sEXPORTED_RUNTIME_METHODS=stringToUTF8,HEAPU8 \
-sENVIRONMENT=web \
-sMODULARIZE=1 \
-sEXPORT_ES6=1"
```
#### Function Exports
To control which functions are exported (avoiding issues with `$` symbols in auto-generated exports), use `RUSTFLAGS`:
```bash
export RUSTFLAGS="-C link-arg=-sEXPORTED_FUNCTIONS=@${SCRIPT_DIR}/exports.txt -C link-arg=-sEXPORT_ALL=0"
```
Where `exports.txt` contains the list of functions to export (one per line, with `_` prefix):
```
_hello
_generate_db
_init
_search_similar_shapes
# ... etc
```
### 3. Build Process
1. **Source Emscripten environment**:
```bash
source /opt/emsdk/emsdk_env.sh
```
2. **Set build environment**:
```bash
source ./_build_env
```
3. **Build**:
```bash
cargo build --target=wasm32-unknown-emscripten
```
## Key Differences from Native Builds
1. **Single-threaded**: WASM builds use `SINGLE_THREADED=TRUE` in CMake
2. **Exception handling**: C++ exceptions are enabled at compile time (`-fexceptions`) and runtime (`-sDISABLE_EXCEPTION_CATCHING=0`)
3. **Linking order**: Libraries are linked after FFI compilation for WASM
4. **Error handling**: C++ exceptions are mapped to `FailedQuery` errors in WASM
5. **Function exports**: Manual control of exported functions via `EXPORTED_FUNCTIONS` file
## Troubleshooting
### Missing Symbols
If you encounter "missing function" errors at runtime, ensure:
- All required static libraries are listed in `link_libraries()` for WASM
- Libraries are linked in the correct order (after FFI compilation)
- `EXPORTED_FUNCTIONS` includes all functions you need to call from JavaScript
### Invalid Export Names
If you see errors like `invalid export name: cxxbridge1$exception`:
- Use `EXPORT_ALL=0` and manually specify functions in `exports.txt`
- Avoid using `EXPORT_ALL=1` with auto-generated export lists that may contain `$` symbols
### CMake Compiler Detection Errors
If CMake fails to detect the compiler:
- Ensure `CC` and `CXX` environment variables point to `emcc` and `em++`
- The Emscripten toolchain should be automatically detected by `cmake-rs`
## References
- [Emscripten Documentation](https://emscripten.org/docs/getting_started/index.html)
- [Rust and WebAssembly](https://rustwasm.github.io/docs/book/)
- [cxx crate documentation](https://cxx.rs/)

105
graph-wasm/_build_env Normal file
View File

@@ -0,0 +1,105 @@
#!/usr/bin/env bash
# --------------------
# Build configuration
# --------------------
_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
export CURRENT_VERSION="${CURRENT_VERSION:-develop}"
export BUILD_NAME="${BUILD_NAME:-graph-wasm}"
export CARGO_BUILD_TARGET="${CARGO_BUILD_TARGET:-wasm32-unknown-emscripten}"
# Keep the emscripten cache in-repo so system cache cleaners do not wipe it.
export EM_CACHE="${EM_CACHE:-${_SCRIPT_DIR}/.emsdk_cache}"
export CARGO_TARGET_DIR="${CARGO_TARGET_DIR:-${_SCRIPT_DIR}/target}"
if [[ -z "${CARGO_INCREMENTAL:-}" && "${NODE_ENV:-}" != "production" ]]; then
export CARGO_INCREMENTAL=1
fi
if [[ -z "${RUSTC_WRAPPER:-}" ]] && command -v sccache >/dev/null 2>&1; then
export RUSTC_WRAPPER=sccache
fi
export CRATE_CC_NO_DEFAULTS=1
# BUILD_MODE
if [[ "${NODE_ENV:-}" == "production" ]]; then
BUILD_MODE=release
else
BUILD_MODE="${1:-debug}"
fi
export BUILD_MODE
# --------------------
# Emscripten memory
# --------------------
export EM_INITIAL_HEAP=$((256 * 1024 * 1024))
export EM_MAXIMUM_MEMORY=$((4 * 1024 * 1024 * 1024))
export EM_MEMORY_GROWTH_GEOMETRIC_STEP=0.8
export EM_MALLOC=dlmalloc
# --------------------
# Flags
# --------------------
EMCC_COMMON_FLAGS=(
--no-entry
-sASSERTIONS=1
-sALLOW_TABLE_GROWTH=1
-sALLOW_MEMORY_GROWTH=1
-sINITIAL_HEAP=$EM_INITIAL_HEAP
-sMEMORY_GROWTH_GEOMETRIC_STEP=$EM_MEMORY_GROWTH_GEOMETRIC_STEP
-sMAXIMUM_MEMORY=$EM_MAXIMUM_MEMORY
-sERROR_ON_UNDEFINED_SYMBOLS=0
-sDISABLE_EXCEPTION_CATCHING=0
-sEXPORT_NAME=createGraphModule
-sEXPORTED_RUNTIME_METHODS=stringToUTF8,HEAPU8
-sENVIRONMENT=web
-sMODULARIZE=1
-sEXPORT_ES6=1
)
export RUSTFLAGS="-C link-arg=-sEXPORTED_FUNCTIONS=@${_SCRIPT_DIR}/exports.txt -C link-arg=-sEXPORT_ALL=0"
# Mode-specific flags
if [[ "$BUILD_MODE" == "release" ]]; then
export EMCC_CFLAGS="-Os ${EMCC_COMMON_FLAGS[*]}"
CARGO_PARAMS=(--release "${@:2}")
else
export EMCC_CFLAGS="-g -sVERBOSE=1 -sMALLOC=$EM_MALLOC ${EMCC_COMMON_FLAGS[*]}"
CARGO_PARAMS=("${@:2}")
fi
export CARGO_PARAMS
# --------------------
# Tasks
# --------------------
clean() {
cargo clean
}
setup() {
:
}
build() {
cargo build "${CARGO_PARAMS[@]}"
}
copy_artifacts() {
local dest=$1
local base="target/$CARGO_BUILD_TARGET/$BUILD_MODE"
mkdir -p "$dest"
cp "$base/graph_wasm.js" "$dest/$BUILD_NAME.js"
cp "$base/graph_wasm.wasm" "$dest/$BUILD_NAME.wasm"
sed -i "s/graph_wasm.wasm/$BUILD_NAME.wasm?version=$CURRENT_VERSION/g" \
"$dest/$BUILD_NAME.js"
}
copy_shared_artifact() {
:
}

20
graph-wasm/build Executable file
View File

@@ -0,0 +1,20 @@
#!/usr/bin/env bash
EMSDK_QUIET=1 . /opt/emsdk/emsdk_env.sh
_SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd);
pushd $_SCRIPT_DIR;
. ./_build_env
set -ex;
setup;
build;
copy_artifacts "../frontend/resources/public/js";
copy_shared_artifact;
exit $?;
popd

26
graph-wasm/build.log Normal file
View File

@@ -0,0 +1,26 @@
~/penpot/graph-wasm ~/penpot/graph-wasm
./_build_env: line 60: export: `CXX_wasm32-unknown-emscripten=/home/penpot/penpot/graph-wasm/wrapper-em++.sh': not a valid identifier
./_build_env: line 110: export: `CXXFLAGS_wasm32-unknown-emscripten=-ffunction-sections -fdata-sections -fexceptions --target=wasm32-unknown-emscripten': not a valid identifier
+ setup
+ true
+ build
+ cargo build
Compiling graph v0.1.0 (/home/penpot/penpot/graph-wasm)
warning: unused import: `Error`
--> src/main.rs:1:48
|
1 | use lbug::{Database, Connection, SystemConfig, Error};
| ^^^^^
|
= note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
warning: variable does not need to be mutable
--> src/main.rs:23:9
|
23 | let mut conn = match Connection::new(&db) {
| ----^^^^
| |
| help: remove this `mut`
|
= note: `#[warn(unused_mut)]` (part of `#[warn(unused)]`) on by default

2
graph-wasm/build.rs Normal file
View File

@@ -0,0 +1,2 @@
// We need this empty script so OUT_DIR is automatically set
fn main() {}

11
graph-wasm/exports.txt Normal file
View File

@@ -0,0 +1,11 @@
_hello
_generate_db
_init
_search_similar_shapes
_free_similar_shapes_buffer
_set_shape_parent
_set_shape_selrect
_set_shape_type
_use_shape

5
graph-wasm/lbug-0.12.2/.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
target/
debug/
**/*.rs.bk

1234
graph-wasm/lbug-0.12.2/Cargo.lock generated Normal file
View File

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,144 @@
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g., crates.io) dependencies.
#
# If you are reading this file be aware that the original Cargo.toml
# will likely look very different (and much more reasonable).
# See Cargo.toml.orig for the original contents.
[package]
edition = "2021"
rust-version = "1.81"
name = "lbug"
version = "0.12.2"
build = "build.rs"
include = [
"build.rs",
"/src",
"/include",
"/lbug-src/src",
"/lbug-src/cmake",
"/lbug-src/third_party",
"/lbug-src/CMakeLists.txt",
"/lbug-src/tools/CMakeLists.txt",
]
autolib = false
autobins = false
autoexamples = false
autotests = false
autobenches = false
description = "An in-process property graph database management system built for query speed and scalability"
homepage = "https://ladybugdb.com/"
readme = "lbug-src/README.md"
keywords = [
"database",
"graph",
"ffi",
]
categories = ["database"]
license = "MIT"
repository = "https://github.com/lbugdb/lbug"
[package.metadata.docs.rs]
all-features = true
[features]
arrow = ["dep:arrow"]
default = []
extension_tests = []
[lib]
name = "lbug"
path = "src/lib.rs"
[dependencies.arrow]
version = "55"
features = ["ffi"]
optional = true
default-features = false
[dependencies.cxx]
version = "=1.0.138"
[dependencies.rust_decimal]
version = "1.37"
default-features = false
[dependencies.time]
version = "0.3"
[dependencies.uuid]
version = "1.6"
[dev-dependencies.anyhow]
version = "1"
[dev-dependencies.rust_decimal_macros]
version = "1.37"
[dev-dependencies.tempfile]
version = "3"
[dev-dependencies.time]
version = "0.3"
features = ["macros"]
[build-dependencies.cmake]
version = "0.1"
[build-dependencies.cxx-build]
version = "=1.0.138"
[build-dependencies.rustversion]
version = "1"
[lints.clippy]
inline_always = "allow"
missing_errors_doc = "allow"
missing_panics_doc = "allow"
module_name_repetitions = "allow"
must_use_candidate = "allow"
needless_pass_by_value = "allow"
redundant_closure_for_method_calls = "allow"
return_self_not_must_use = "allow"
similar_names = "allow"
struct_excessive_bools = "allow"
too_many_arguments = "allow"
too_many_lines = "allow"
type_complexity = "allow"
unreadable_literal = "allow"
[lints.clippy.cargo]
level = "warn"
priority = -1
[lints.clippy.complexity]
level = "warn"
priority = -1
[lints.clippy.correctness]
level = "warn"
priority = -1
[lints.clippy.pedantic]
level = "warn"
priority = -1
[lints.clippy.perf]
level = "warn"
priority = -1
[lints.clippy.style]
level = "warn"
priority = -1
[lints.clippy.suspicious]
level = "warn"
priority = -1
[profile.relwithdebinfo]
debug = 2
inherits = "release"

View File

@@ -0,0 +1,283 @@
use std::env;
use std::path::{Path, PathBuf};
fn link_mode() -> &'static str {
if env::var("LBUG_SHARED").is_ok() {
"dylib"
} else {
"static"
}
}
fn get_target() -> String {
env::var("PROFILE").unwrap()
}
fn is_wasm_emscripten() -> bool {
env::var("TARGET")
.map(|t| t == "wasm32-unknown-emscripten")
.unwrap_or(false)
}
fn link_libraries() {
// For wasm32-unknown-emscripten, we need to link lbug and all its dependencies
// These are built by CMake and need to be linked here
if is_wasm_emscripten() {
// Link all dependencies first (built by CMake)
for lib in [
"utf8proc",
"antlr4_cypher",
"antlr4_runtime",
"re2",
"fastpfor",
"parquet",
"thrift",
"snappy",
"zstd",
"miniz",
"mbedtls",
"brotlidec",
"brotlicommon",
"lz4",
"roaring_bitmap",
"simsimd",
] {
println!("cargo:rustc-link-lib=static={lib}");
}
// Link the lbug static library (built by CMake)
println!("cargo:rustc-link-lib=static=lbug");
// Don't link system libraries for wasm (they're handled by Emscripten)
return;
}
// This also needs to be set by any crates using it if they want to use extensions
if !cfg!(windows) && link_mode() == "static" {
println!("cargo:rustc-link-arg=-rdynamic");
}
if cfg!(windows) && link_mode() == "dylib" {
println!("cargo:rustc-link-lib=dylib=lbug_shared");
} else if link_mode() == "dylib" {
println!("cargo:rustc-link-lib={}=lbug", link_mode());
} else if rustversion::cfg!(since(1.82)) {
println!("cargo:rustc-link-lib=static:+whole-archive=lbug");
} else {
println!("cargo:rustc-link-lib=static=lbug");
}
if link_mode() == "static" {
if cfg!(windows) {
println!("cargo:rustc-link-lib=dylib=msvcrt");
println!("cargo:rustc-link-lib=dylib=shell32");
println!("cargo:rustc-link-lib=dylib=ole32");
} else if cfg!(target_os = "macos") {
println!("cargo:rustc-link-lib=dylib=c++");
} else {
println!("cargo:rustc-link-lib=dylib=stdc++");
}
for lib in [
"utf8proc",
"antlr4_cypher",
"antlr4_runtime",
"re2",
"fastpfor",
"parquet",
"thrift",
"snappy",
"zstd",
"miniz",
"mbedtls",
"brotlidec",
"brotlicommon",
"lz4",
"roaring_bitmap",
"simsimd",
] {
if rustversion::cfg!(since(1.82)) {
println!("cargo:rustc-link-lib=static:+whole-archive={lib}");
} else {
println!("cargo:rustc-link-lib=static={lib}");
}
}
}
}
fn build_bundled_cmake() -> Vec<PathBuf> {
let lbug_root = {
let root = Path::new(&std::env::var("CARGO_MANIFEST_DIR").unwrap()).join("lbug-src");
if root.is_symlink() || root.is_dir() {
root
} else {
// If the path is not directory, this is probably an in-source build on windows where the
// symlink is unreadable.
Path::new(&std::env::var("CARGO_MANIFEST_DIR").unwrap()).join("../..")
}
};
let mut build = cmake::Config::new(&lbug_root);
build
.no_build_target(true)
.define("BUILD_SHELL", "OFF")
.define("BUILD_SINGLE_FILE_HEADER", "OFF")
.define("AUTO_UPDATE_GRAMMAR", "OFF");
// Configure for wasm32-unknown-emscripten
if is_wasm_emscripten() {
// Same configuration as ladybug/tools/wasm build
build.define("SINGLE_THREADED", "TRUE");
// cmake-rs should automatically detect emscripten toolchain when CC/CXX point to emcc/em++
} else if cfg!(windows) {
build.generator("Ninja");
build.cxxflag("/EHsc");
build.define("CMAKE_MSVC_RUNTIME_LIBRARY", "MultiThreadedDLL");
build.define("CMAKE_POLICY_DEFAULT_CMP0091", "NEW");
}
if let Ok(jobs) = env::var("NUM_JOBS") {
// SAFETY: Setting environment variables in build scripts is safe
unsafe {
env::set_var("CMAKE_BUILD_PARALLEL_LEVEL", jobs);
}
}
let build_dir = build.build();
let lbug_lib_path = build_dir.join("build").join("src");
println!("cargo:rustc-link-search=native={}", lbug_lib_path.display());
for dir in [
"utf8proc",
"antlr4_cypher",
"antlr4_runtime",
"re2",
"brotli",
"alp",
"fastpfor",
"parquet",
"thrift",
"snappy",
"zstd",
"miniz",
"mbedtls",
"lz4",
"roaring_bitmap",
"simsimd",
] {
let lib_path = build_dir
.join("build")
.join("third_party")
.join(dir)
.canonicalize()
.unwrap_or_else(|_| {
panic!(
"Could not find {}/build/third_party/{}",
build_dir.display(),
dir
)
});
println!("cargo:rustc-link-search=native={}", lib_path.display());
}
vec![
lbug_root.join("src/include"),
build_dir.join("build/src"),
build_dir.join("build/src/include"),
lbug_root.join("third_party/nlohmann_json"),
lbug_root.join("third_party/fastpfor"),
lbug_root.join("third_party/alp/include"),
]
}
fn build_ffi(
bridge_file: &str,
out_name: &str,
source_file: &str,
bundled: bool,
include_paths: &Vec<PathBuf>,
) {
let mut build = cxx_build::bridge(bridge_file);
build.file(source_file);
if bundled {
build.define("LBUG_BUNDLED", None);
}
if get_target() == "debug" || get_target() == "relwithdebinfo" {
build.define("ENABLE_RUNTIME_CHECKS", "1");
}
if link_mode() == "static" {
build.define("LBUG_STATIC_DEFINE", None);
}
build.includes(include_paths);
println!("cargo:rerun-if-env-changed=LBUG_SHARED");
println!("cargo:rerun-if-changed=include/lbug_rs.h");
println!("cargo:rerun-if-changed=src/lbug_rs.cpp");
// Note that this should match the lbug-src/* entries in the package.include list in Cargo.toml
// Unfortunately they appear to need to be specified individually since the symlink is
// considered to be changed each time.
println!("cargo:rerun-if-changed=lbug-src/src");
println!("cargo:rerun-if-changed=lbug-src/cmake");
println!("cargo:rerun-if-changed=lbug-src/third_party");
println!("cargo:rerun-if-changed=lbug-src/CMakeLists.txt");
println!("cargo:rerun-if-changed=lbug-src/tools/CMakeLists.txt");
if is_wasm_emscripten() {
// For emscripten, use C++20 and enable exceptions
build.flag("-std=c++20");
build.flag("-fexceptions");
// Note: -sDISABLE_EXCEPTION_CATCHING=0 is a linker flag, not a compiler flag
// It should be set via EMCC_CFLAGS environment variable or cargo rustc-link-arg
} else if cfg!(windows) {
build.flag("/std:c++20");
build.flag("/MD");
} else {
build.flag("-std=c++2a");
}
build.compile(out_name);
}
fn main() {
if env::var("DOCS_RS").is_ok() {
// Do nothing; we're just building docs and don't need the C++ library
return;
}
let mut bundled = false;
let mut include_paths =
vec![Path::new(&std::env::var("CARGO_MANIFEST_DIR").unwrap()).join("include")];
if let (Ok(lbug_lib_dir), Ok(lbug_include)) =
(env::var("LBUG_LIBRARY_DIR"), env::var("LBUG_INCLUDE_DIR"))
{
println!("cargo:rustc-link-search=native={lbug_lib_dir}");
println!("cargo:rustc-link-arg=-Wl,-rpath,{lbug_lib_dir}");
include_paths.push(Path::new(&lbug_include).to_path_buf());
} else {
include_paths.extend(build_bundled_cmake());
bundled = true;
}
// For wasm, we need to link libraries after building FFI to ensure proper symbol resolution
if !is_wasm_emscripten() && link_mode() == "static" {
link_libraries();
}
build_ffi(
"src/ffi.rs",
"lbug_rs",
"src/lbug_rs.cpp",
bundled,
&include_paths,
);
if cfg!(feature = "arrow") {
build_ffi(
"src/ffi/arrow.rs",
"lbug_arrow_rs",
"src/lbug_arrow.cpp",
bundled,
&include_paths,
);
}
// For wasm, link libraries after FFI; for dylib, link after FFI
if is_wasm_emscripten() || link_mode() == "dylib" {
link_libraries();
}
}

View File

@@ -0,0 +1,15 @@
#pragma once
#include "rust/cxx.h"
#ifdef LBUG_BUNDLED
#include "main/lbug.h"
#else
#include <lbug.hpp>
#endif
namespace lbug_arrow {
ArrowSchema query_result_get_arrow_schema(const lbug::main::QueryResult& result);
ArrowArray query_result_get_next_arrow_chunk(lbug::main::QueryResult& result, uint64_t chunkSize);
} // namespace lbug_arrow

View File

@@ -0,0 +1,243 @@
#pragma once
#include <cstdint>
#include <memory>
#include "rust/cxx.h"
#ifdef LBUG_BUNDLED
#include "common/type_utils.h"
#include "common/types/int128_t.h"
#include "common/types/types.h"
#include "common/types/value/nested.h"
#include "common/types/value/node.h"
#include "common/types/value/recursive_rel.h"
#include "common/types/value/rel.h"
#include "common/types/value/value.h"
#include "main/lbug.h"
#include "storage/storage_version_info.h"
#else
#include <lbug.hpp>
#endif
namespace lbug_rs {
struct TypeListBuilder {
std::vector<lbug::common::LogicalType> types;
void insert(std::unique_ptr<lbug::common::LogicalType> type) {
types.push_back(std::move(*type));
}
};
std::unique_ptr<TypeListBuilder> create_type_list();
struct QueryParams {
std::unordered_map<std::string, std::unique_ptr<lbug::common::Value>> inputParams;
void insert(const rust::Str key, std::unique_ptr<lbug::common::Value> value) {
inputParams.insert(std::make_pair(key, std::move(value)));
}
};
std::unique_ptr<QueryParams> new_params();
std::unique_ptr<lbug::common::LogicalType> create_logical_type(lbug::common::LogicalTypeID id);
std::unique_ptr<lbug::common::LogicalType> create_logical_type_list(
std::unique_ptr<lbug::common::LogicalType> childType);
std::unique_ptr<lbug::common::LogicalType> create_logical_type_array(
std::unique_ptr<lbug::common::LogicalType> childType, uint64_t numElements);
inline std::unique_ptr<lbug::common::LogicalType> create_logical_type_struct(
const rust::Vec<rust::String>& fieldNames, std::unique_ptr<TypeListBuilder> fieldTypes) {
std::vector<lbug::common::StructField> fields;
for (auto i = 0u; i < fieldNames.size(); i++) {
fields.emplace_back(std::string(fieldNames[i]), std::move(fieldTypes->types[i]));
}
return std::make_unique<lbug::common::LogicalType>(
lbug::common::LogicalType::STRUCT(std::move(fields)));
}
inline std::unique_ptr<lbug::common::LogicalType> create_logical_type_union(
const rust::Vec<rust::String>& fieldNames, std::unique_ptr<TypeListBuilder> fieldTypes) {
std::vector<lbug::common::StructField> fields;
for (auto i = 0u; i < fieldNames.size(); i++) {
fields.emplace_back(std::string(fieldNames[i]), std::move(fieldTypes->types[i]));
}
return std::make_unique<lbug::common::LogicalType>(
lbug::common::LogicalType::UNION(std::move(fields)));
}
std::unique_ptr<lbug::common::LogicalType> create_logical_type_map(
std::unique_ptr<lbug::common::LogicalType> keyType,
std::unique_ptr<lbug::common::LogicalType> valueType);
inline std::unique_ptr<lbug::common::LogicalType> create_logical_type_decimal(uint32_t precision,
uint32_t scale) {
return std::make_unique<lbug::common::LogicalType>(
lbug::common::LogicalType::DECIMAL(precision, scale));
}
std::unique_ptr<lbug::common::LogicalType> logical_type_get_list_child_type(
const lbug::common::LogicalType& logicalType);
std::unique_ptr<lbug::common::LogicalType> logical_type_get_array_child_type(
const lbug::common::LogicalType& logicalType);
uint64_t logical_type_get_array_num_elements(const lbug::common::LogicalType& logicalType);
rust::Vec<rust::String> logical_type_get_struct_field_names(const lbug::common::LogicalType& value);
std::unique_ptr<std::vector<lbug::common::LogicalType>> logical_type_get_struct_field_types(
const lbug::common::LogicalType& value);
inline uint32_t logical_type_get_decimal_precision(const lbug::common::LogicalType& logicalType) {
return lbug::common::DecimalType::getPrecision(logicalType);
}
inline uint32_t logical_type_get_decimal_scale(const lbug::common::LogicalType& logicalType) {
return lbug::common::DecimalType::getScale(logicalType);
}
/* Database */
std::unique_ptr<lbug::main::Database> new_database(std::string_view databasePath,
uint64_t bufferPoolSize, uint64_t maxNumThreads, bool enableCompression, bool readOnly,
uint64_t maxDBSize, bool autoCheckpoint, int64_t checkpointThreshold,
bool throwOnWalReplayFailure, bool enableChecksums);
void database_set_logging_level(lbug::main::Database& database, const std::string& level);
/* Connection */
std::unique_ptr<lbug::main::Connection> database_connect(lbug::main::Database& database);
std::unique_ptr<lbug::main::QueryResult> connection_execute(lbug::main::Connection& connection,
lbug::main::PreparedStatement& query, std::unique_ptr<QueryParams> params);
inline std::unique_ptr<lbug::main::QueryResult> connection_query(lbug::main::Connection& connection,
std::string_view query) {
return connection.query(query);
}
/* PreparedStatement */
rust::String prepared_statement_error_message(const lbug::main::PreparedStatement& statement);
/* QueryResult */
rust::String query_result_to_string(const lbug::main::QueryResult& result);
rust::String query_result_get_error_message(const lbug::main::QueryResult& result);
double query_result_get_compiling_time(const lbug::main::QueryResult& result);
double query_result_get_execution_time(const lbug::main::QueryResult& result);
std::unique_ptr<std::vector<lbug::common::LogicalType>> query_result_column_data_types(
const lbug::main::QueryResult& query_result);
rust::Vec<rust::String> query_result_column_names(const lbug::main::QueryResult& query_result);
/* NodeVal/RelVal */
rust::String node_value_get_label_name(const lbug::common::Value& val);
rust::String rel_value_get_label_name(const lbug::common::Value& val);
size_t node_value_get_num_properties(const lbug::common::Value& value);
size_t rel_value_get_num_properties(const lbug::common::Value& value);
rust::String node_value_get_property_name(const lbug::common::Value& value, size_t index);
rust::String rel_value_get_property_name(const lbug::common::Value& value, size_t index);
const lbug::common::Value& node_value_get_property_value(const lbug::common::Value& value,
size_t index);
const lbug::common::Value& rel_value_get_property_value(const lbug::common::Value& value,
size_t index);
/* NodeVal */
const lbug::common::Value& node_value_get_node_id(const lbug::common::Value& val);
/* RelVal */
const lbug::common::Value& rel_value_get_src_id(const lbug::common::Value& val);
std::array<uint64_t, 2> rel_value_get_dst_id(const lbug::common::Value& val);
/* RecursiveRel */
const lbug::common::Value& recursive_rel_get_nodes(const lbug::common::Value& val);
const lbug::common::Value& recursive_rel_get_rels(const lbug::common::Value& val);
/* FlatTuple */
const lbug::common::Value& flat_tuple_get_value(const lbug::processor::FlatTuple& flatTuple,
uint32_t index);
/* Value */
const std::string& value_get_string(const lbug::common::Value& value);
template<typename T>
std::unique_ptr<T> value_get_unique(const lbug::common::Value& value) {
return std::make_unique<T>(value.getValue<T>());
}
int64_t value_get_interval_secs(const lbug::common::Value& value);
int32_t value_get_interval_micros(const lbug::common::Value& value);
int32_t value_get_date_days(const lbug::common::Value& value);
int64_t value_get_timestamp_ns(const lbug::common::Value& value);
int64_t value_get_timestamp_ms(const lbug::common::Value& value);
int64_t value_get_timestamp_sec(const lbug::common::Value& value);
int64_t value_get_timestamp_micros(const lbug::common::Value& value);
int64_t value_get_timestamp_tz(const lbug::common::Value& value);
std::array<uint64_t, 2> value_get_int128_t(const lbug::common::Value& value);
std::array<uint64_t, 2> value_get_internal_id(const lbug::common::Value& value);
uint32_t value_get_children_size(const lbug::common::Value& value);
const lbug::common::Value& value_get_child(const lbug::common::Value& value, uint32_t index);
lbug::common::LogicalTypeID value_get_data_type_id(const lbug::common::Value& value);
const lbug::common::LogicalType& value_get_data_type(const lbug::common::Value& value);
inline lbug::common::PhysicalTypeID value_get_physical_type(const lbug::common::Value& value) {
return value.getDataType().getPhysicalType();
}
rust::String value_to_string(const lbug::common::Value& val);
std::unique_ptr<lbug::common::Value> create_value_string(lbug::common::LogicalTypeID typ,
const rust::Slice<const unsigned char> value);
std::unique_ptr<lbug::common::Value> create_value_timestamp(const int64_t timestamp);
std::unique_ptr<lbug::common::Value> create_value_timestamp_tz(const int64_t timestamp);
std::unique_ptr<lbug::common::Value> create_value_timestamp_ns(const int64_t timestamp);
std::unique_ptr<lbug::common::Value> create_value_timestamp_ms(const int64_t timestamp);
std::unique_ptr<lbug::common::Value> create_value_timestamp_sec(const int64_t timestamp);
inline std::unique_ptr<lbug::common::Value> create_value_date(const int32_t date) {
return std::make_unique<lbug::common::Value>(lbug::common::date_t(date));
}
std::unique_ptr<lbug::common::Value> create_value_interval(const int32_t months, const int32_t days,
const int64_t micros);
std::unique_ptr<lbug::common::Value> create_value_null(
std::unique_ptr<lbug::common::LogicalType> typ);
std::unique_ptr<lbug::common::Value> create_value_int128_t(int64_t high, uint64_t low);
std::unique_ptr<lbug::common::Value> create_value_internal_id(uint64_t offset, uint64_t table);
inline std::unique_ptr<lbug::common::Value> create_value_uuid_t(int64_t high, uint64_t low) {
return std::make_unique<lbug::common::Value>(
lbug::common::ku_uuid_t{lbug::common::int128_t(low, high)});
}
template<typename T>
std::unique_ptr<lbug::common::Value> create_value(const T value) {
return std::make_unique<lbug::common::Value>(value);
}
inline std::unique_ptr<lbug::common::Value> create_value_decimal(int64_t high, uint64_t low,
uint32_t scale, uint32_t precision) {
auto value =
std::make_unique<lbug::common::Value>(lbug::common::LogicalType::DECIMAL(precision, scale),
std::vector<std::unique_ptr<lbug::common::Value>>{});
auto i128 = lbug::common::int128_t(low, high);
lbug::common::TypeUtils::visit(
value->getDataType().getPhysicalType(),
[&](lbug::common::int128_t) { value->val.int128Val = i128; },
[&](int64_t) { value->val.int64Val = static_cast<int64_t>(i128); },
[&](int32_t) { value->val.int32Val = static_cast<int32_t>(i128); },
[&](int16_t) { value->val.int16Val = static_cast<int16_t>(i128); },
[](auto) { KU_UNREACHABLE; });
return value;
}
struct ValueListBuilder {
std::vector<std::unique_ptr<lbug::common::Value>> values;
void insert(std::unique_ptr<lbug::common::Value> value) { values.push_back(std::move(value)); }
};
std::unique_ptr<lbug::common::Value> get_list_value(std::unique_ptr<lbug::common::LogicalType> typ,
std::unique_ptr<ValueListBuilder> value);
std::unique_ptr<ValueListBuilder> create_list();
inline std::string_view string_view_from_str(rust::Str s) {
return {s.data(), s.size()};
}
inline lbug::storage::storage_version_t get_storage_version() {
return lbug::storage::StorageVersionInfo::getStorageVersion();
}
} // namespace lbug_rs

View File

@@ -0,0 +1,454 @@
cmake_minimum_required(VERSION 3.15)
project(Lbug VERSION 0.12.2 LANGUAGES CXX C)
option(SINGLE_THREADED "Single-threaded mode" FALSE)
if(SINGLE_THREADED)
set(__SINGLE_THREADED__ TRUE)
add_compile_definitions(__SINGLE_THREADED__)
message(STATUS "Single-threaded mode is enabled")
else()
message(STATUS "Multi-threaded mode is enabled: CMAKE_BUILD_PARALLEL_LEVEL=$ENV{CMAKE_BUILD_PARALLEL_LEVEL}")
find_package(Threads REQUIRED)
endif()
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED TRUE)
set(CMAKE_CXX_VISIBILITY_PRESET hidden)
set(CMAKE_C_VISIBILITY_PRESET hidden)
set(CMAKE_EXPORT_COMPILE_COMMANDS TRUE)
set(CMAKE_FIND_PACKAGE_RESOLVE_SYMLINKS TRUE)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_VISIBILITY_INLINES_HIDDEN ON)
# On Linux, symbols in executables are not accessible by loaded shared libraries (e.g. via dlopen(3)). However, we need to export public symbols in executables so that extensions can access public symbols. This enables that behaviour.
set(CMAKE_ENABLE_EXPORTS TRUE)
option(ENABLE_WERROR "Treat all warnings as errors" FALSE)
if(ENABLE_WERROR)
if (CMAKE_VERSION VERSION_GREATER "3.24.0" OR CMAKE_VERSION VERSION_EQUAL "3.24.0")
set(CMAKE_COMPILE_WARNING_AS_ERROR TRUE)
elseif (MSVC)
add_compile_options(\WX)
else ()
add_compile_options(-Werror)
endif()
endif()
# Detect OS and architecture, copied from DuckDB
set(OS_NAME "unknown")
set(OS_ARCH "amd64")
string(REGEX MATCH "(arm64|aarch64)" IS_ARM "${CMAKE_SYSTEM_PROCESSOR}")
if(IS_ARM)
set(OS_ARCH "arm64")
elseif(FORCE_32_BIT)
set(OS_ARCH "i386")
endif()
if(APPLE)
set(OS_NAME "osx")
endif()
if(WIN32)
set(OS_NAME "windows")
endif()
if(UNIX AND NOT APPLE)
set(OS_NAME "linux") # sorry BSD
endif()
if(CMAKE_SIZEOF_VOID_P EQUAL 8)
message(STATUS "64-bit architecture detected")
add_compile_definitions(__64BIT__)
elseif(CMAKE_SIZEOF_VOID_P EQUAL 4)
message(STATUS "32-bit architecture detected")
add_compile_definitions(__32BIT__)
set(__32BIT__ TRUE)
endif()
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release)
endif()
if(DEFINED ENV{PYBIND11_PYTHON_VERSION})
set(PYBIND11_PYTHON_VERSION $ENV{PYBIND11_PYTHON_VERSION})
endif()
if(DEFINED ENV{PYTHON_EXECUTABLE})
set(PYTHON_EXECUTABLE $ENV{PYTHON_EXECUTABLE})
endif()
find_program(CCACHE_PROGRAM ccache)
if (CCACHE_PROGRAM)
set(CMAKE_C_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
set(CMAKE_CXX_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
message(STATUS "ccache found and enabled")
else ()
find_program(CCACHE_PROGRAM sccache)
if (CCACHE_PROGRAM)
set(CMAKE_C_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
set(CMAKE_CXX_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
message(STATUS "sccache found and enabled")
endif ()
endif ()
set(INSTALL_LIB_DIR
lib
CACHE PATH "Installation directory for libraries")
set(INSTALL_BIN_DIR
bin
CACHE PATH "Installation directory for executables")
set(INSTALL_INCLUDE_DIR
include
CACHE PATH "Installation directory for header files")
set(INSTALL_CMAKE_DIR
${DEF_INSTALL_CMAKE_DIR}
CACHE PATH "Installation directory for CMake files")
option(ENABLE_ADDRESS_SANITIZER "Enable address sanitizer." FALSE)
option(ENABLE_THREAD_SANITIZER "Enable thread sanitizer." FALSE)
option(ENABLE_UBSAN "Enable undefined behavior sanitizer." FALSE)
option(ENABLE_RUNTIME_CHECKS "Enable runtime coherency checks (e.g. asserts)" FALSE)
option(ENABLE_LTO "Enable Link-Time Optimization" FALSE)
option(ENABLE_MALLOC_BUFFER_MANAGER "Enable Buffer manager using malloc. Default option for webassembly" OFF)
option(LBUG_DEFAULT_REL_STORAGE_DIRECTION "Only store fwd direction in rel tables by default." BOTH)
if(NOT LBUG_DEFAULT_REL_STORAGE_DIRECTION)
set(LBUG_DEFAULT_REL_STORAGE_DIRECTION BOTH)
endif()
set(LBUG_DEFAULT_REL_STORAGE_DIRECTION ${LBUG_DEFAULT_REL_STORAGE_DIRECTION}_REL_STORAGE)
option(LBUG_PAGE_SIZE_LOG2 "Log2 of the page size." 12)
if(NOT LBUG_PAGE_SIZE_LOG2)
set(LBUG_PAGE_SIZE_LOG2 12)
endif()
message(STATUS "LBUG_PAGE_SIZE_LOG2: ${LBUG_PAGE_SIZE_LOG2}")
option(LBUG_VECTOR_CAPACITY_LOG2 "Log2 of the vector capacity." 11)
if(NOT LBUG_VECTOR_CAPACITY_LOG2)
set(LBUG_VECTOR_CAPACITY_LOG2 11)
endif()
message(STATUS "LBUG_VECTOR_CAPACITY_LOG2: ${LBUG_VECTOR_CAPACITY_LOG2}")
# 64 * 2048 nodes per group
option(LBUG_NODE_GROUP_SIZE_LOG2 "Log2 of the vector capacity." 17)
if(NOT LBUG_NODE_GROUP_SIZE_LOG2)
set(LBUG_NODE_GROUP_SIZE_LOG2 17)
endif()
message(STATUS "LBUG_NODE_GROUP_SIZE_LOG2: ${LBUG_NODE_GROUP_SIZE_LOG2}")
option(LBUG_MAX_SEGMENT_SIZE_LOG2 "Log2 of the maximum segment size in bytes." 18)
if(NOT LBUG_MAX_SEGMENT_SIZE_LOG2)
set(LBUG_MAX_SEGMENT_SIZE_LOG2 18)
endif()
message(STATUS "LBUG_MAX_SEGMENT_SIZE_LOG2: ${LBUG_MAX_SEGMENT_SIZE_LOG2}")
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/cmake/templates/system_config.h.in ${CMAKE_CURRENT_BINARY_DIR}/src/include/common/system_config.h @ONLY)
include(CheckCXXSymbolExists)
check_cxx_symbol_exists(F_FULLFSYNC "fcntl.h" HAS_FULLFSYNC)
check_cxx_symbol_exists(fdatasync "unistd.h" HAS_FDATASYNC)
if(HAS_FULLFSYNC)
message(STATUS "✓ F_FULLFSYNC will be used on this platform")
add_compile_definitions(HAS_FULLFSYNC)
else()
message(STATUS "✗ F_FULLFSYNC not available")
endif()
if(HAS_FDATASYNC)
message(STATUS "✓ fdatasync will be used on this platform")
add_compile_definitions(HAS_FDATASYNC)
else()
message(STATUS "✗ fdatasync not available, using fsync fallback")
endif()
if(MSVC)
# Required for M_PI on Windows
add_compile_definitions(_USE_MATH_DEFINES)
add_compile_definitions(NOMINMAX)
add_compile_definitions(SERD_STATIC)
# This is a workaround for regex oom issue on windows in gtest.
add_compile_definitions(_REGEX_MAX_STACK_COUNT=0)
add_compile_definitions(_REGEX_MAX_COMPLEXITY_COUNT=0)
# Disable constexpr mutex constructor to avoid compatibility issues with
# older versions of the MSVC runtime library
# See: https://github.com/microsoft/STL/wiki/Changelog#vs-2022-1710
add_compile_definitions(_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR)
# TODO (bmwinger): Figure out if this can be set automatically by cmake,
# or at least better integrated with user-specified options
# For now, hardcode _AMD64_
# CMAKE_GENERATOR_PLATFORM can be used for visual studio builds, but not for ninja
add_compile_definitions(_AMD64_)
# Non-english windows system may use other encodings other than utf-8 (e.g. Chinese use GBK).
add_compile_options("/utf-8")
# Enables support for custom hardware exception handling
add_compile_options("/EHa")
# Reduces the size of the static library by roughly 1/2
add_compile_options("/Zc:inline")
# Disable type conversion warnings
add_compile_options(/wd4244 /wd4267)
# Remove the default to avoid warnings
STRING(REPLACE "/EHsc" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
STRING(REPLACE "/EHs" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
# Store all libraries and binaries in the same directory so that lbug_shared.dll is found at runtime
set(LIBRARY_OUTPUT_PATH "${CMAKE_BINARY_DIR}/src")
set(EXECUTABLE_OUTPUT_PATH "${CMAKE_BINARY_DIR}/src")
# This is a workaround for regex stackoverflow issue on windows in gtest.
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /STACK:8388608")
string(REGEX REPLACE "/W[3|4]" "/w" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
add_compile_options($<$<CONFIG:Release>:/W0>)
else()
add_compile_options(-Wall -Wextra)
# Disable warnings for unknown pragmas, which is used by several third-party libraries
add_compile_options(-Wno-unknown-pragmas)
endif()
if(${BUILD_WASM})
if(NOT __SINGLE_THREADED__)
add_compile_options(-pthread)
add_link_options(-pthread)
add_link_options(-sPTHREAD_POOL_SIZE=8)
endif()
add_compile_options(-s DISABLE_EXCEPTION_CATCHING=0)
add_link_options(-sSTACK_SIZE=4MB)
add_link_options(-sASSERTIONS=1)
add_link_options(-lembind)
add_link_options(-sWASM_BIGINT)
if(BUILD_TESTS OR BUILD_EXTENSION_TESTS)
add_link_options(-sINITIAL_MEMORY=3892MB)
add_link_options(-sNODERAWFS=1)
elseif(WASM_NODEFS)
add_link_options(-sNODERAWFS=1)
add_link_options(-sALLOW_MEMORY_GROWTH=1)
add_link_options(-sMODULARIZE=1)
add_link_options(-sEXPORTED_RUNTIME_METHODS=FS,wasmMemory)
add_link_options(-sEXPORT_NAME=lbug)
add_link_options(-sMAXIMUM_MEMORY=4GB)
else()
add_link_options(-sSINGLE_FILE=1)
add_link_options(-sALLOW_MEMORY_GROWTH=1)
add_link_options(-sMODULARIZE=1)
add_link_options(-sEXPORTED_RUNTIME_METHODS=FS,wasmMemory)
add_link_options(-lidbfs.js)
add_link_options(-lworkerfs.js)
add_link_options(-sEXPORT_NAME=lbug)
add_link_options(-sMAXIMUM_MEMORY=4GB)
endif()
set(__WASM__ TRUE)
add_compile_options(-fexceptions)
add_link_options(-s DISABLE_EXCEPTION_CATCHING=0)
add_link_options(-fexceptions)
add_compile_definitions(__WASM__)
set(ENABLE_MALLOC_BUFFER_MANAGER ON)
endif()
if(${BUILD_SWIFT})
add_compile_definitions(__SWIFT__)
set(ENABLE_MALLOC_BUFFER_MANAGER ON)
endif()
if (${ENABLE_MALLOC_BUFFER_MANAGER})
add_compile_definitions(BM_MALLOC)
endif()
if(ANDROID_ABI)
message(STATUS "Android ABI detected: ${ANDROID_ABI}")
add_compile_definitions(__ANDROID__)
set(__ANDROID__ TRUE)
endif()
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
add_compile_options(-Wno-restrict) # no restrict until https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105651 is fixed
endif()
if(${ENABLE_THREAD_SANITIZER} AND (NOT __SINGLE_THREADED__))
if(MSVC)
message(FATAL_ERROR "Thread sanitizer is not supported on MSVC")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=thread -fno-omit-frame-pointer")
endif()
endif()
if(${ENABLE_ADDRESS_SANITIZER})
if(MSVC)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /fsanitize=address")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fno-omit-frame-pointer")
endif()
endif()
if(${ENABLE_UBSAN})
if(MSVC)
message(FATAL_ERROR "Undefined behavior sanitizer is not supported on MSVC")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=undefined -fno-omit-frame-pointer")
endif()
endif()
if(${ENABLE_RUNTIME_CHECKS})
add_compile_definitions(LBUG_RUNTIME_CHECKS)
endif()
if (${ENABLE_DESER_DEBUG})
add_compile_definitions(LBUG_DESER_DEBUG)
endif()
if(${ENABLE_LTO})
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE)
endif()
option(AUTO_UPDATE_GRAMMAR "Automatically regenerate C++ grammar files on change." TRUE)
option(BUILD_BENCHMARK "Build benchmarks." FALSE)
option(BUILD_EXTENSIONS "Semicolon-separated list of extensions to build." "")
option(BUILD_EXAMPLES "Build examples." FALSE)
option(BUILD_JAVA "Build Java API." FALSE)
option(BUILD_NODEJS "Build NodeJS API." FALSE)
option(BUILD_PYTHON "Build Python API." FALSE)
option(BUILD_SHELL "Build Interactive Shell" TRUE)
option(BUILD_SINGLE_FILE_HEADER "Build single file header. Requires Python >= 3.9." TRUE)
option(BUILD_TESTS "Build C++ tests." FALSE)
option(BUILD_EXTENSION_TESTS "Build C++ extension tests." FALSE)
option(BUILD_LBUG "Build Lbug." TRUE)
option(ENABLE_BACKTRACES "Enable backtrace printing for exceptions and segfaults" FALSE)
option(USE_STD_FORMAT "Use std::format instead of a custom formatter." FALSE)
option(PREFER_SYSTEM_DEPS "Only download certain deps if not found on the system" TRUE)
option(BUILD_LCOV "Build coverage report." FALSE)
if(${BUILD_LCOV})
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage")
endif()
if (ENABLE_BACKTRACES)
set(DOWNLOAD_CPPTRACE TRUE)
if(${PREFER_SYSTEM_DEPS})
find_package(cpptrace QUIET)
if(cpptrace_FOUND)
message(STATUS "Using system cpptrace")
set(DOWNLOAD_CPPTRACE FALSE)
endif()
endif()
if(${DOWNLOAD_CPPTRACE})
message(STATUS "Fetching cpptrace from GitHub...")
include(FetchContent)
FetchContent_Declare(
cpptrace
GIT_REPOSITORY https://github.com/jeremy-rifkin/cpptrace.git
GIT_TAG v0.8.3
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(cpptrace)
endif()
add_compile_definitions(LBUG_BACKTRACE)
endif()
if (USE_STD_FORMAT)
add_compile_definitions(USE_STD_FORMAT)
endif()
function(add_lbug_test TEST_NAME)
set(SRCS ${ARGN})
add_executable(${TEST_NAME} ${SRCS})
target_link_libraries(${TEST_NAME} PRIVATE test_helper test_runner graph_test)
if (ENABLE_BACKTRACES)
target_link_libraries(${TEST_NAME} PRIVATE register_backtrace_signal_handler)
endif()
target_include_directories(${TEST_NAME} PRIVATE ${PROJECT_SOURCE_DIR}/test/include)
include(GoogleTest)
if (TEST_NAME STREQUAL "e2e_test")
gtest_discover_tests(${TEST_NAME}
DISCOVERY_TIMEOUT 600
DISCOVERY_MODE PRE_TEST
TEST_PREFIX e2e_test_
)
else()
gtest_discover_tests(${TEST_NAME}
DISCOVERY_TIMEOUT 600
DISCOVERY_MODE PRE_TEST
)
endif()
endfunction()
function(add_lbug_api_test TEST_NAME)
set(SRCS ${ARGN})
add_executable(${TEST_NAME} ${SRCS})
target_link_libraries(${TEST_NAME} PRIVATE api_graph_test api_test_helper)
if (ENABLE_BACKTRACES)
target_link_libraries(${TEST_NAME} PRIVATE register_backtrace_signal_handler)
endif()
target_include_directories(${TEST_NAME} PRIVATE ${PROJECT_SOURCE_DIR}/test/include)
include(GoogleTest)
gtest_discover_tests(${TEST_NAME})
endfunction()
# Windows doesn't support dynamic lookup, so we have to link extensions against lbug.
if (MSVC AND (NOT BUILD_EXTENSIONS EQUAL ""))
set(BUILD_LBUG TRUE)
endif ()
include_directories(third_party/antlr4_cypher/include)
include_directories(third_party/antlr4_runtime/src)
include_directories(third_party/brotli/c/include)
include_directories(third_party/fast_float/include)
include_directories(third_party/mbedtls/include)
include_directories(third_party/parquet)
include_directories(third_party/snappy)
include_directories(third_party/thrift)
include_directories(third_party/miniz)
include_directories(third_party/nlohmann_json)
include_directories(third_party/pybind11/include)
include_directories(third_party/pyparse)
include_directories(third_party/re2/include)
include_directories(third_party/alp/include)
if (${BUILD_TESTS} OR ${BUILD_EXTENSION_TESTS})
include_directories(third_party/spdlog)
elseif (${BUILD_BENCHMARK})
include_directories(third_party/spdlog)
endif ()
include_directories(third_party/utf8proc/include)
include_directories(third_party/zstd/include)
include_directories(third_party/httplib)
include_directories(third_party/pcg)
include_directories(third_party/lz4)
include_directories(third_party/roaring_bitmap)
# Use SYSTEM to suppress warnings from simsimd
include_directories(SYSTEM third_party/simsimd/include)
add_subdirectory(third_party)
add_definitions(-DLBUG_ROOT_DIRECTORY="${PROJECT_SOURCE_DIR}")
add_definitions(-DLBUG_CMAKE_VERSION="${CMAKE_PROJECT_VERSION}")
add_definitions(-DLBUG_EXTENSION_VERSION="0.12.0")
if(BUILD_LBUG)
include_directories(
src/include
${CMAKE_CURRENT_BINARY_DIR}/src/include
)
endif()
if (EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/extension/CMakeLists.txt")
add_subdirectory(extension)
endif ()
if(BUILD_LBUG)
add_subdirectory(src)
# Link extensions which require static linking.
foreach(ext IN LISTS STATICALLY_LINKED_EXTENSIONS)
if (${BUILD_EXTENSION_TESTS})
add_compile_definitions(__STATIC_LINK_EXTENSION_TEST__)
endif ()
target_link_libraries(lbug PRIVATE "lbug_${ext}_static_extension")
target_link_libraries(lbug_shared PRIVATE "lbug_${ext}_static_extension")
endforeach()
if (${BUILD_TESTS} OR ${BUILD_EXTENSION_TESTS})
add_subdirectory(test)
elseif (${BUILD_BENCHMARK})
add_subdirectory(test/test_helper)
endif ()
add_subdirectory(tools)
endif ()
if (${BUILD_EXAMPLES})
add_subdirectory(examples/c)
add_subdirectory(examples/cpp)
endif()

View File

@@ -0,0 +1,73 @@
<div align="center">
<picture>
<!-- <source srcset="https://ladybugdb.com/img/lbug-logo-dark.png" media="(prefers-color-scheme: dark)"> -->
<img src="https://ladybugdb.com/logo.png" height="100" alt="Ladybug Logo">
</picture>
</div>
<br>
<p align="center">
<a href="https://github.com/LadybugDB/ladybug/actions">
<img src="https://github.com/LadybugDB/ladybug/actions/workflows/ci-workflow.yml/badge.svg?branch=master" alt="Github Actions Badge"></a>
<a href="https://discord.com/invite/hXyHmvW3Vy">
<img src="https://img.shields.io/discord/1162999022819225631?logo=discord" alt="discord" /></a>
<a href="https://twitter.com/lbugdb">
<img src="https://img.shields.io/badge/follow-@lbugdb-1DA1F2?logo=twitter" alt="twitter"></a>
</p>
# Ladybug
Ladybug is an embedded graph database built for query speed and scalability. Ladybug is optimized for handling complex analytical workloads
on very large databases and provides a set of retrieval features, such as a full text search and vector indices. Our core feature set includes:
- Flexible Property Graph Data Model and Cypher query language
- Embeddable, serverless integration into applications
- Native full text search and vector index
- Columnar disk-based storage
- Columnar sparse row-based (CSR) adjacency list/join indices
- Vectorized and factorized query processor
- Novel and very fast join algorithms
- Multi-core query parallelism
- Serializable ACID transactions
- Wasm (WebAssembly) bindings for fast, secure execution in the browser
Ladybug is being developed by [LadybugDB Developers](https://github.com/LadybugDB) and
is available under a permissible license. So try it out and help us make it better! We welcome your feedback and feature requests.
The database was formerly known as [Kuzu](https://github.com/kuzudb/kuzu).
## Installation
> [!WARNING]
> Many of these binary installation methods are not functional yet. We need to work through package names, availability and convention issues.
> For now, use the build from source method.
| Language | Installation |
| -------- |------------------------------------------------------------------------|
| Python | `pip install real_ladybug` |
| NodeJS | `npm install lbug` |
| Rust | `cargo add lbug` |
| Go | `go get github.com/lbugdb/go-lbug` |
| Swift | [lbug-swift](https://github.com/lbugdb/lbug-swift) |
| Java | [Maven Central](https://central.sonatype.com/artifact/com.ladybugdb/lbug) |
| C/C++ | [precompiled binaries](https://github.com/LadybugDB/ladybug/releases/latest) |
| CLI | [precompiled binaries](https://github.com/LadybugDB/ladybug/releases/latest) |
To learn more about installation, see our [Installation](https://docs.ladybugdb.com/installation) page.
## Getting Started
Refer to our [Getting Started](https://docs.ladybugdb.com/get-started/) page for your first example.
## Build from Source
You can build from source using the instructions provided in the [developer guide](https://docs.ladybugdb.com/developer-guide/).
## Contributing
We welcome contributions to Ladybug. If you are interested in contributing to Ladybug, please read our [Contributing Guide](CONTRIBUTING.md).
## License
By contributing to Ladybug, you agree that your contributions will be licensed under the [MIT License](LICENSE).
## Contact
You can contact us at [social@ladybugdb.com](mailto:social@ladybugdb.com) or [join our Discord community](https://discord.com/invite/hXyHmvW3Vy).

View File

@@ -0,0 +1,75 @@
/*
* This is a template header used for generating the header 'system_config.h'
* Any value in the format @VALUE_NAME@ can be substituted with a value passed into CMakeLists.txt
* See https://cmake.org/cmake/help/latest/command/configure_file.html for more details
*/
#pragma once
#include <algorithm>
#include <cstdint>
#include "common/enums/extend_direction.h"
#define BOTH_REL_STORAGE 0
#define FWD_REL_STORAGE 1
#define BWD_REL_STORAGE 2
namespace lbug {
namespace common {
#define VECTOR_CAPACITY_LOG_2 @LBUG_VECTOR_CAPACITY_LOG2@
#if VECTOR_CAPACITY_LOG_2 > 12
#error "Vector capacity log2 should be less than or equal to 12"
#endif
constexpr uint64_t DEFAULT_VECTOR_CAPACITY = static_cast<uint64_t>(1) << VECTOR_CAPACITY_LOG_2;
// Currently the system supports files with 2 different pages size, which we refer to as
// PAGE_SIZE and TEMP_PAGE_SIZE. PAGE_SIZE is the default size of the page which is the
// unit of read/write to the database files.
static constexpr uint64_t PAGE_SIZE_LOG2 = @LBUG_PAGE_SIZE_LOG2@; // Default to 4KB.
static constexpr uint64_t LBUG_PAGE_SIZE = static_cast<uint64_t>(1) << PAGE_SIZE_LOG2;
// Page size for files with large pages, e.g., temporary files that are used by operators that
// may require large amounts of memory.
static constexpr uint64_t TEMP_PAGE_SIZE_LOG2 = 18;
static const uint64_t TEMP_PAGE_SIZE = static_cast<uint64_t>(1) << TEMP_PAGE_SIZE_LOG2;
#define DEFAULT_REL_STORAGE_DIRECTION @LBUG_DEFAULT_REL_STORAGE_DIRECTION@
#if DEFAULT_REL_STORAGE_DIRECTION == FWD_REL_STORAGE
static constexpr ExtendDirection DEFAULT_EXTEND_DIRECTION = ExtendDirection::FWD;
#elif DEFAULT_REL_STORAGE_DIRECTION == BWD_REL_STORAGE
static constexpr ExtendDirection DEFAULT_EXTEND_DIRECTION = ExtendDirection::BWD;
#else
static constexpr ExtendDirection DEFAULT_EXTEND_DIRECTION = ExtendDirection::BOTH;
#endif
struct StorageConfig {
static constexpr uint64_t NODE_GROUP_SIZE_LOG2 = @LBUG_NODE_GROUP_SIZE_LOG2@;
static constexpr uint64_t NODE_GROUP_SIZE = static_cast<uint64_t>(1) << NODE_GROUP_SIZE_LOG2;
// The number of CSR lists in a leaf region.
static constexpr uint64_t CSR_LEAF_REGION_SIZE_LOG2 =
std::min(static_cast<uint64_t>(10), NODE_GROUP_SIZE_LOG2 - 1);
static constexpr uint64_t CSR_LEAF_REGION_SIZE = static_cast<uint64_t>(1)
<< CSR_LEAF_REGION_SIZE_LOG2;
static constexpr uint64_t CHUNKED_NODE_GROUP_CAPACITY =
std::min(static_cast<uint64_t>(2048), NODE_GROUP_SIZE);
// Maximum size for a segment in bytes
static constexpr uint64_t MAX_SEGMENT_SIZE_LOG2 = @LBUG_MAX_SEGMENT_SIZE_LOG2@;
static constexpr uint64_t MAX_SEGMENT_SIZE = 1 << MAX_SEGMENT_SIZE_LOG2;
};
struct OrderByConfig {
static constexpr uint64_t MIN_SIZE_TO_REDUCE = common::DEFAULT_VECTOR_CAPACITY * 5;
};
struct CopyConfig {
static constexpr uint64_t PANDAS_PARTITION_COUNT = 50 * DEFAULT_VECTOR_CAPACITY;
};
} // namespace common
} // namespace lbug
#undef BOTH_REL_STORAGE
#undef FWD_REL_STORAGE
#undef BWD_REL_STORAGE

View File

@@ -0,0 +1,79 @@
include_directories(${CMAKE_CURRENT_BINARY_DIR})
# Have to pass this down to every subdirectory, which actually adds the files.
# This doesn't affect parent directories.
add_compile_definitions(LBUG_EXPORTS)
add_compile_definitions(ANTLR4CPP_STATIC)
add_subdirectory(binder)
add_subdirectory(c_api)
add_subdirectory(catalog)
add_subdirectory(common)
add_subdirectory(expression_evaluator)
add_subdirectory(function)
add_subdirectory(graph)
add_subdirectory(main)
add_subdirectory(optimizer)
add_subdirectory(parser)
add_subdirectory(planner)
add_subdirectory(processor)
add_subdirectory(storage)
add_subdirectory(transaction)
add_subdirectory(extension)
add_library(lbug STATIC ${ALL_OBJECT_FILES})
add_library(lbug_shared SHARED ${ALL_OBJECT_FILES})
set(LBUG_LIBRARIES antlr4_cypher antlr4_runtime brotlidec brotlicommon fast_float utf8proc re2 fastpfor parquet snappy thrift yyjson zstd miniz mbedtls lz4 roaring_bitmap simsimd)
if (NOT __SINGLE_THREADED__)
set(LBUG_LIBRARIES ${LBUG_LIBRARIES} Threads::Threads)
endif()
if(NOT WIN32)
set(LBUG_LIBRARIES dl ${LBUG_LIBRARIES})
endif()
# Seems to be needed for clang on linux only
# for compiling std::atomic<T>::compare_exchange_weak
if ((NOT APPLE AND CMAKE_CXX_COMPILER_ID STREQUAL "Clang") AND NOT __WASM__ AND NOT __SINGLE_THREADED__)
set(LBUG_LIBRARIES atomic ${LBUG_LIBRARIES})
endif()
if (ENABLE_BACKTRACES)
set(LBUG_LIBRARIES ${LBUG_LIBRARIES} cpptrace::cpptrace)
endif()
target_link_libraries(lbug PUBLIC ${LBUG_LIBRARIES})
target_link_libraries(lbug_shared PUBLIC ${LBUG_LIBRARIES})
unset(LBUG_LIBRARIES)
set(LBUG_INCLUDES $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include> $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}> ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/include/c_api ${CMAKE_CURRENT_BINARY_DIR}/../src/include)
target_include_directories(lbug PUBLIC ${LBUG_INCLUDES})
target_include_directories(lbug_shared PUBLIC ${LBUG_INCLUDES})
unset(LBUG_INCLUDES)
if(WIN32)
# Anything linking against the static library must not use dllimport.
target_compile_definitions(lbug INTERFACE LBUG_STATIC_DEFINE)
endif()
if(NOT WIN32)
set_target_properties(lbug_shared PROPERTIES OUTPUT_NAME lbug)
endif()
install(TARGETS lbug lbug_shared)
if(${BUILD_SINGLE_FILE_HEADER})
# Create a command to generate lbug.hpp, and then create a target that is
# always built that depends on it. This allows our generator to detect when
# exactly to build lbug.hpp, while still building the target by default.
find_package(Python3 3.9...4 REQUIRED)
add_custom_command(
OUTPUT lbug.hpp
COMMAND
${Python3_EXECUTABLE} ${PROJECT_SOURCE_DIR}/scripts/collect-single-file-header.py ${CMAKE_CURRENT_BINARY_DIR}/..
DEPENDS
${PROJECT_SOURCE_DIR}/scripts/collect-single-file-header.py lbug_shared)
add_custom_target(single_file_header ALL DEPENDS lbug.hpp)
endif()
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/include/c_api/lbug.h TYPE INCLUDE)
if(${BUILD_SINGLE_FILE_HEADER})
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/lbug.hpp TYPE INCLUDE)
endif()

View File

@@ -0,0 +1,917 @@
ku_Statements
: oC_Cypher ( SP? ';' SP? oC_Cypher )* SP? EOF ;
oC_Cypher
: oC_AnyCypherOption? SP? ( oC_Statement ) ( SP? ';' )?;
oC_Statement
: oC_Query
| kU_CreateUser
| kU_CreateRole
| kU_CreateNodeTable
| kU_CreateRelTable
| kU_CreateSequence
| kU_CreateType
| kU_Drop
| kU_AlterTable
| kU_CopyFrom
| kU_CopyFromByColumn
| kU_CopyTO
| kU_StandaloneCall
| kU_CreateMacro
| kU_CommentOn
| kU_Transaction
| kU_Extension
| kU_ExportDatabase
| kU_ImportDatabase
| kU_AttachDatabase
| kU_DetachDatabase
| kU_UseDatabase;
kU_CopyFrom
: COPY SP oC_SchemaName kU_ColumnNames? SP FROM SP kU_ScanSource ( SP? '(' SP? kU_Options SP? ')' )? ;
kU_ColumnNames
: SP? '(' SP? (oC_SchemaName ( SP? ',' SP? oC_SchemaName )* SP?)? ')';
kU_ScanSource
: kU_FilePaths
| '(' SP? oC_Query SP? ')'
| oC_Parameter
| oC_Variable
| oC_Variable '.' SP? oC_SchemaName
| oC_FunctionInvocation ;
kU_CopyFromByColumn
: COPY SP oC_SchemaName SP FROM SP '(' SP? StringLiteral ( SP? ',' SP? StringLiteral )* ')' SP BY SP COLUMN ;
kU_CopyTO
: COPY SP '(' SP? oC_Query SP? ')' SP TO SP StringLiteral ( SP? '(' SP? kU_Options SP? ')' )? ;
kU_ExportDatabase
: EXPORT SP DATABASE SP StringLiteral ( SP? '(' SP? kU_Options SP? ')' )? ;
kU_ImportDatabase
: IMPORT SP DATABASE SP StringLiteral;
kU_AttachDatabase
: ATTACH SP StringLiteral (SP AS SP oC_SchemaName)? SP '(' SP? DBTYPE SP oC_SymbolicName (SP? ',' SP? kU_Options)? SP? ')' ;
kU_Option
: oC_SymbolicName (SP? '=' SP? | SP*) oC_Literal | oC_SymbolicName;
kU_Options
: kU_Option ( SP? ',' SP? kU_Option )* ;
kU_DetachDatabase
: DETACH SP oC_SchemaName;
kU_UseDatabase
: USE SP oC_SchemaName;
kU_StandaloneCall
: CALL SP oC_SymbolicName SP? '=' SP? oC_Expression
| CALL SP oC_FunctionInvocation;
kU_CommentOn
: COMMENT SP ON SP TABLE SP oC_SchemaName SP IS SP StringLiteral ;
kU_CreateMacro
: CREATE SP MACRO SP oC_FunctionName SP? '(' SP? kU_PositionalArgs? SP? kU_DefaultArg? ( SP? ',' SP? kU_DefaultArg )* SP? ')' SP AS SP oC_Expression ;
kU_PositionalArgs
: oC_SymbolicName ( SP? ',' SP? oC_SymbolicName )* ;
kU_DefaultArg
: oC_SymbolicName SP? ':' '=' SP? oC_Literal ;
kU_FilePaths
: '[' SP? StringLiteral ( SP? ',' SP? StringLiteral )* ']'
| StringLiteral
| GLOB SP? '(' SP? StringLiteral SP? ')' ;
kU_IfNotExists
: IF SP NOT SP EXISTS ;
kU_CreateNodeTable
: CREATE SP NODE SP TABLE SP (kU_IfNotExists SP)? oC_SchemaName ( SP? '(' SP? kU_PropertyDefinitions SP? ( ',' SP? kU_CreateNodeConstraint )? SP? ')' | SP AS SP oC_Query ) ;
kU_CreateRelTable
: CREATE SP REL SP TABLE ( SP GROUP )? ( SP kU_IfNotExists )? SP oC_SchemaName
SP? '(' SP?
kU_FromToConnections SP? (
( ',' SP? kU_PropertyDefinitions SP? )?
( ',' SP? oC_SymbolicName SP? )? // Constraints
')'
| ')' SP AS SP oC_Query )
( SP WITH SP? '(' SP? kU_Options SP? ')')? ;
kU_FromToConnections
: kU_FromToConnection ( SP? ',' SP? kU_FromToConnection )* ;
kU_FromToConnection
: FROM SP oC_SchemaName SP TO SP oC_SchemaName ;
kU_CreateSequence
: CREATE SP SEQUENCE SP (kU_IfNotExists SP)? oC_SchemaName (SP kU_SequenceOptions)* ;
kU_CreateType
: CREATE SP TYPE SP oC_SchemaName SP AS SP kU_DataType SP? ;
kU_SequenceOptions
: kU_IncrementBy
| kU_MinValue
| kU_MaxValue
| kU_StartWith
| kU_Cycle;
kU_WithPasswd
: SP WITH SP PASSWORD SP StringLiteral ;
kU_CreateUser
: CREATE SP USER SP (kU_IfNotExists SP)? oC_Variable kU_WithPasswd? ;
kU_CreateRole
: CREATE SP ROLE SP (kU_IfNotExists SP)? oC_Variable ;
kU_IncrementBy : INCREMENT SP ( BY SP )? MINUS? oC_IntegerLiteral ;
kU_MinValue : (NO SP MINVALUE) | (MINVALUE SP MINUS? oC_IntegerLiteral) ;
kU_MaxValue : (NO SP MAXVALUE) | (MAXVALUE SP MINUS? oC_IntegerLiteral) ;
kU_StartWith : START SP ( WITH SP )? MINUS? oC_IntegerLiteral ;
kU_Cycle : (NO SP)? CYCLE ;
kU_IfExists
: IF SP EXISTS ;
kU_Drop
: DROP SP (TABLE | SEQUENCE | MACRO) SP (kU_IfExists SP)? oC_SchemaName ;
kU_AlterTable
: ALTER SP TABLE SP oC_SchemaName SP kU_AlterOptions ;
kU_AlterOptions
: kU_AddProperty
| kU_DropProperty
| kU_RenameTable
| kU_RenameProperty
| kU_AddFromToConnection
| kU_DropFromToConnection;
kU_AddProperty
: ADD SP (kU_IfNotExists SP)? oC_PropertyKeyName SP kU_DataType ( SP kU_Default )? ;
kU_Default
: DEFAULT SP oC_Expression ;
kU_DropProperty
: DROP SP (kU_IfExists SP)? oC_PropertyKeyName ;
kU_RenameTable
: RENAME SP TO SP oC_SchemaName ;
kU_RenameProperty
: RENAME SP oC_PropertyKeyName SP TO SP oC_PropertyKeyName ;
kU_AddFromToConnection
: ADD SP (kU_IfNotExists SP)? kU_FromToConnection ;
kU_DropFromToConnection
: DROP SP (kU_IfExists SP)? kU_FromToConnection ;
kU_ColumnDefinitions: kU_ColumnDefinition ( SP? ',' SP? kU_ColumnDefinition )* ;
kU_ColumnDefinition : oC_PropertyKeyName SP kU_DataType ;
kU_PropertyDefinitions : kU_PropertyDefinition ( SP? ',' SP? kU_PropertyDefinition )* ;
kU_PropertyDefinition : kU_ColumnDefinition ( SP kU_Default )? ( SP PRIMARY SP KEY)?;
kU_CreateNodeConstraint : PRIMARY SP KEY SP? '(' SP? oC_PropertyKeyName SP? ')' ;
DECIMAL: ( 'D' | 'd' ) ( 'E' | 'e' ) ( 'C' | 'c' ) ( 'I' | 'i' ) ( 'M' | 'm' ) ( 'A' | 'a' ) ( 'L' | 'l' ) ;
kU_UnionType
: UNION SP? '(' SP? kU_ColumnDefinitions SP? ')' ;
kU_StructType
: STRUCT SP? '(' SP? kU_ColumnDefinitions SP? ')' ;
kU_MapType
: MAP SP? '(' SP? kU_DataType SP? ',' SP? kU_DataType SP? ')' ;
kU_DecimalType
: DECIMAL SP? '(' SP? oC_IntegerLiteral SP? ',' SP? oC_IntegerLiteral SP? ')' ;
kU_DataType
: oC_SymbolicName
| kU_DataType kU_ListIdentifiers
| kU_UnionType
| kU_StructType
| kU_MapType
| kU_DecimalType ;
kU_ListIdentifiers : kU_ListIdentifier ( kU_ListIdentifier )* ;
kU_ListIdentifier : '[' oC_IntegerLiteral? ']' ;
oC_AnyCypherOption
: oC_Explain
| oC_Profile ;
oC_Explain
: EXPLAIN (SP LOGICAL)? ;
oC_Profile
: PROFILE ;
kU_Transaction
: BEGIN SP TRANSACTION
| BEGIN SP TRANSACTION SP READ SP ONLY
| COMMIT
| ROLLBACK
| CHECKPOINT;
kU_Extension
: kU_LoadExtension
| kU_InstallExtension
| kU_UninstallExtension
| kU_UpdateExtension ;
kU_LoadExtension
: LOAD SP (EXTENSION SP)? ( StringLiteral | oC_Variable ) ;
kU_InstallExtension
: (FORCE SP)? INSTALL SP oC_Variable (SP FROM SP StringLiteral)?;
kU_UninstallExtension
: UNINSTALL SP oC_Variable;
kU_UpdateExtension
: UPDATE SP oC_Variable;
oC_Query
: oC_RegularQuery ;
oC_RegularQuery
: oC_SingleQuery ( SP? oC_Union )*
| (oC_Return SP? )+ oC_SingleQuery { notifyReturnNotAtEnd($ctx->start); }
;
oC_Union
: ( UNION SP ALL SP? oC_SingleQuery )
| ( UNION SP? oC_SingleQuery ) ;
oC_SingleQuery
: oC_SinglePartQuery
| oC_MultiPartQuery
;
oC_SinglePartQuery
: ( oC_ReadingClause SP? )* oC_Return
| ( ( oC_ReadingClause SP? )* oC_UpdatingClause ( SP? oC_UpdatingClause )* ( SP? oC_Return )? )
;
oC_MultiPartQuery
: ( kU_QueryPart SP? )+ oC_SinglePartQuery;
kU_QueryPart
: (oC_ReadingClause SP? )* ( oC_UpdatingClause SP? )* oC_With ;
oC_UpdatingClause
: oC_Create
| oC_Merge
| oC_Set
| oC_Delete
;
oC_ReadingClause
: oC_Match
| oC_Unwind
| kU_InQueryCall
| kU_LoadFrom
;
kU_LoadFrom
: LOAD ( SP WITH SP HEADERS SP? '(' SP? kU_ColumnDefinitions SP? ')' )? SP FROM SP kU_ScanSource (SP? '(' SP? kU_Options SP? ')')? (SP? oC_Where)? ;
oC_YieldItem
: ( oC_Variable SP AS SP )? oC_Variable ;
oC_YieldItems
: oC_YieldItem ( SP? ',' SP? oC_YieldItem )* ;
kU_InQueryCall
: CALL SP oC_FunctionInvocation (SP? oC_Where)? ( SP? YIELD SP oC_YieldItems )? ;
oC_Match
: ( OPTIONAL SP )? MATCH SP? oC_Pattern ( SP oC_Where )? ( SP kU_Hint )? ;
kU_Hint
: HINT SP kU_JoinNode;
kU_JoinNode
: kU_JoinNode SP JOIN SP kU_JoinNode
| kU_JoinNode ( SP MULTI_JOIN SP oC_SchemaName)+
| '(' SP? kU_JoinNode SP? ')'
| oC_SchemaName ;
oC_Unwind : UNWIND SP? oC_Expression SP AS SP oC_Variable ;
oC_Create
: CREATE SP? oC_Pattern ;
// For unknown reason, openCypher use oC_PatternPart instead of oC_Pattern. There should be no difference in terms of planning.
// So we choose to be consistent with oC_Create and use oC_Pattern instead.
oC_Merge : MERGE SP? oC_Pattern ( SP oC_MergeAction )* ;
oC_MergeAction
: ( ON SP MATCH SP oC_Set )
| ( ON SP CREATE SP oC_Set )
;
oC_Set
: SET SP? oC_SetItem ( SP? ',' SP? oC_SetItem )*
| SET SP? oC_Atom SP? '=' SP? kU_Properties;
oC_SetItem
: ( oC_PropertyExpression SP? '=' SP? oC_Expression ) ;
oC_Delete
: ( DETACH SP )? DELETE SP? oC_Expression ( SP? ',' SP? oC_Expression )*;
oC_With
: WITH oC_ProjectionBody ( SP? oC_Where )? ;
oC_Return
: RETURN oC_ProjectionBody ;
oC_ProjectionBody
: ( SP? DISTINCT )? SP oC_ProjectionItems (SP oC_Order )? ( SP oC_Skip )? ( SP oC_Limit )? ;
oC_ProjectionItems
: ( STAR ( SP? ',' SP? oC_ProjectionItem )* )
| ( oC_ProjectionItem ( SP? ',' SP? oC_ProjectionItem )* )
;
STAR : '*' ;
oC_ProjectionItem
: ( oC_Expression SP AS SP oC_Variable )
| oC_Expression
;
oC_Order
: ORDER SP BY SP oC_SortItem ( ',' SP? oC_SortItem )* ;
oC_Skip
: L_SKIP SP oC_Expression ;
L_SKIP : ( 'S' | 's' ) ( 'K' | 'k' ) ( 'I' | 'i' ) ( 'P' | 'p' ) ;
oC_Limit
: LIMIT SP oC_Expression ;
oC_SortItem
: oC_Expression ( SP? ( ASCENDING | ASC | DESCENDING | DESC ) )? ;
oC_Where
: WHERE SP oC_Expression ;
oC_Pattern
: oC_PatternPart ( SP? ',' SP? oC_PatternPart )* ;
oC_PatternPart
: ( oC_Variable SP? '=' SP? oC_AnonymousPatternPart )
| oC_AnonymousPatternPart ;
oC_AnonymousPatternPart
: oC_PatternElement ;
oC_PatternElement
: ( oC_NodePattern ( SP? oC_PatternElementChain )* )
| ( '(' oC_PatternElement ')' )
;
oC_NodePattern
: '(' SP? ( oC_Variable SP? )? ( oC_NodeLabels SP? )? ( kU_Properties SP? )? ')' ;
oC_PatternElementChain
: oC_RelationshipPattern SP? oC_NodePattern ;
oC_RelationshipPattern
: ( oC_LeftArrowHead SP? oC_Dash SP? oC_RelationshipDetail? SP? oC_Dash )
| ( oC_Dash SP? oC_RelationshipDetail? SP? oC_Dash SP? oC_RightArrowHead )
| ( oC_Dash SP? oC_RelationshipDetail? SP? oC_Dash )
;
oC_RelationshipDetail
: '[' SP? ( oC_Variable SP? )? ( oC_RelationshipTypes SP? )? ( kU_RecursiveDetail SP? )? ( kU_Properties SP? )? ']' ;
// The original oC_Properties definition is oC_MapLiteral | oC_Parameter.
// We choose to not support parameter as properties which will be the decision for a long time.
// We then substitute with oC_MapLiteral definition. We create oC_MapLiteral only when we decide to add MAP type.
kU_Properties
: '{' SP? ( oC_PropertyKeyName SP? ':' SP? oC_Expression SP? ( ',' SP? oC_PropertyKeyName SP? ':' SP? oC_Expression SP? )* )? '}';
oC_RelationshipTypes
: ':' SP? oC_RelTypeName ( SP? '|' ':'? SP? oC_RelTypeName )* ;
oC_NodeLabels
: ':' SP? oC_LabelName ( SP? ('|' ':'? | ':') SP? oC_LabelName )* ;
kU_RecursiveDetail
: '*' ( SP? kU_RecursiveType)? ( SP? oC_RangeLiteral )? ( SP? kU_RecursiveComprehension )? ;
kU_RecursiveType
: (ALL SP)? WSHORTEST SP? '(' SP? oC_PropertyKeyName SP? ')'
| SHORTEST
| ALL SP SHORTEST
| TRAIL
| ACYCLIC ;
oC_RangeLiteral
: oC_LowerBound? SP? DOTDOT SP? oC_UpperBound?
| oC_IntegerLiteral ;
kU_RecursiveComprehension
: '(' SP? oC_Variable SP? ',' SP? oC_Variable ( SP? '|' SP? oC_Where SP? )? ( SP? '|' SP? kU_RecursiveProjectionItems SP? ',' SP? kU_RecursiveProjectionItems SP? )? ')' ;
kU_RecursiveProjectionItems
: '{' SP? oC_ProjectionItems? SP? '}' ;
oC_LowerBound
: DecimalInteger ;
oC_UpperBound
: DecimalInteger ;
oC_LabelName
: oC_SchemaName ;
oC_RelTypeName
: oC_SchemaName ;
oC_Expression
: oC_OrExpression ;
oC_OrExpression
: oC_XorExpression ( SP OR SP oC_XorExpression )* ;
oC_XorExpression
: oC_AndExpression ( SP XOR SP oC_AndExpression )* ;
oC_AndExpression
: oC_NotExpression ( SP AND SP oC_NotExpression )* ;
oC_NotExpression
: ( NOT SP? )* oC_ComparisonExpression;
oC_ComparisonExpression
: kU_BitwiseOrOperatorExpression ( SP? kU_ComparisonOperator SP? kU_BitwiseOrOperatorExpression )?
| kU_BitwiseOrOperatorExpression ( SP? INVALID_NOT_EQUAL SP? kU_BitwiseOrOperatorExpression ) { notifyInvalidNotEqualOperator($INVALID_NOT_EQUAL); }
| kU_BitwiseOrOperatorExpression SP? kU_ComparisonOperator SP? kU_BitwiseOrOperatorExpression ( SP? kU_ComparisonOperator SP? kU_BitwiseOrOperatorExpression )+ { notifyNonBinaryComparison($ctx->start); }
;
kU_ComparisonOperator : '=' | '<>' | '<' | '<=' | '>' | '>=' ;
INVALID_NOT_EQUAL : '!=' ;
kU_BitwiseOrOperatorExpression
: kU_BitwiseAndOperatorExpression ( SP? '|' SP? kU_BitwiseAndOperatorExpression )* ;
kU_BitwiseAndOperatorExpression
: kU_BitShiftOperatorExpression ( SP? '&' SP? kU_BitShiftOperatorExpression )* ;
kU_BitShiftOperatorExpression
: oC_AddOrSubtractExpression ( SP? kU_BitShiftOperator SP? oC_AddOrSubtractExpression )* ;
kU_BitShiftOperator : '>>' | '<<' ;
oC_AddOrSubtractExpression
: oC_MultiplyDivideModuloExpression ( SP? kU_AddOrSubtractOperator SP? oC_MultiplyDivideModuloExpression )* ;
kU_AddOrSubtractOperator : '+' | '-' ;
oC_MultiplyDivideModuloExpression
: oC_PowerOfExpression ( SP? kU_MultiplyDivideModuloOperator SP? oC_PowerOfExpression )* ;
kU_MultiplyDivideModuloOperator : '*' | '/' | '%' ;
oC_PowerOfExpression
: oC_StringListNullOperatorExpression ( SP? '^' SP? oC_StringListNullOperatorExpression )* ;
oC_StringListNullOperatorExpression
: oC_UnaryAddSubtractOrFactorialExpression ( oC_StringOperatorExpression | oC_ListOperatorExpression+ | oC_NullOperatorExpression )? ;
oC_ListOperatorExpression
: ( SP IN SP? oC_PropertyOrLabelsExpression )
| ( '[' oC_Expression ']' )
| ( '[' oC_Expression? ( COLON | DOTDOT ) oC_Expression? ']' ) ;
COLON : ':' ;
DOTDOT : '..' ;
oC_StringOperatorExpression
: ( oC_RegularExpression | ( SP STARTS SP WITH ) | ( SP ENDS SP WITH ) | ( SP CONTAINS ) ) SP? oC_PropertyOrLabelsExpression ;
oC_RegularExpression
: SP? '=~' ;
oC_NullOperatorExpression
: ( SP IS SP NULL )
| ( SP IS SP NOT SP NULL ) ;
MINUS : '-' ;
FACTORIAL : '!' ;
oC_UnaryAddSubtractOrFactorialExpression
: ( MINUS SP? )* oC_PropertyOrLabelsExpression (SP? FACTORIAL)? ;
oC_PropertyOrLabelsExpression
: oC_Atom ( SP? oC_PropertyLookup )* ;
oC_Atom
: oC_Literal
| oC_Parameter
| oC_CaseExpression
| oC_ParenthesizedExpression
| oC_FunctionInvocation
| oC_PathPatterns
| oC_ExistCountSubquery
| oC_Variable
| oC_Quantifier
;
oC_Quantifier
: ( ALL SP? '(' SP? oC_FilterExpression SP? ')' )
| ( ANY SP? '(' SP? oC_FilterExpression SP? ')' )
| ( NONE SP? '(' SP? oC_FilterExpression SP? ')' )
| ( SINGLE SP? '(' SP? oC_FilterExpression SP? ')' )
;
oC_FilterExpression
: oC_IdInColl SP oC_Where ;
oC_IdInColl
: oC_Variable SP IN SP oC_Expression ;
oC_Literal
: oC_NumberLiteral
| StringLiteral
| oC_BooleanLiteral
| NULL
| oC_ListLiteral
| kU_StructLiteral
;
oC_BooleanLiteral
: TRUE
| FALSE
;
oC_ListLiteral
: '[' SP? ( oC_Expression SP? ( kU_ListEntry SP? )* )? ']' ;
kU_ListEntry
: ',' SP? oC_Expression? ;
kU_StructLiteral
: '{' SP? kU_StructField SP? ( ',' SP? kU_StructField SP? )* '}' ;
kU_StructField
: ( oC_SymbolicName | StringLiteral ) SP? ':' SP? oC_Expression ;
oC_ParenthesizedExpression
: '(' SP? oC_Expression SP? ')' ;
oC_FunctionInvocation
: COUNT SP? '(' SP? '*' SP? ')'
| CAST SP? '(' SP? kU_FunctionParameter SP? ( ( AS SP? kU_DataType ) | ( ',' SP? kU_FunctionParameter ) ) SP? ')'
| oC_FunctionName SP? '(' SP? ( DISTINCT SP? )? ( kU_FunctionParameter SP? ( ',' SP? kU_FunctionParameter SP? )* )? ')' ;
oC_FunctionName
: oC_SymbolicName ;
kU_FunctionParameter
: ( oC_SymbolicName SP? ':' '=' SP? )? oC_Expression
| kU_LambdaParameter ;
kU_LambdaParameter
: kU_LambdaVars SP? '-' '>' SP? oC_Expression SP? ;
kU_LambdaVars
: oC_SymbolicName
| '(' SP? oC_SymbolicName SP? ( ',' SP? oC_SymbolicName SP?)* ')' ;
oC_PathPatterns
: oC_NodePattern ( SP? oC_PatternElementChain )+;
oC_ExistCountSubquery
: (EXISTS | COUNT) SP? '{' SP? MATCH SP? oC_Pattern ( SP? oC_Where )? ( SP? kU_Hint )? SP? '}' ;
oC_PropertyLookup
: '.' SP? ( oC_PropertyKeyName | STAR ) ;
oC_CaseExpression
: ( ( CASE ( SP? oC_CaseAlternative )+ ) | ( CASE SP? oC_Expression ( SP? oC_CaseAlternative )+ ) ) ( SP? ELSE SP? oC_Expression )? SP? END ;
oC_CaseAlternative
: WHEN SP? oC_Expression SP? THEN SP? oC_Expression ;
oC_Variable
: oC_SymbolicName ;
StringLiteral
: ( '"' ( StringLiteral_0 | EscapedChar )* '"' )
| ( '\'' ( StringLiteral_1 | EscapedChar )* '\'' )
;
EscapedChar
: '\\' ( '\\' | '\'' | '"' | ( 'B' | 'b' ) | ( 'F' | 'f' ) | ( 'N' | 'n' ) | ( 'R' | 'r' ) | ( 'T' | 't' ) | ( ( 'X' | 'x' ) ( HexDigit HexDigit ) ) | ( ( 'U' | 'u' ) ( HexDigit HexDigit HexDigit HexDigit ) ) | ( ( 'U' | 'u' ) ( HexDigit HexDigit HexDigit HexDigit HexDigit HexDigit HexDigit HexDigit ) ) ) ;
oC_NumberLiteral
: oC_DoubleLiteral
| oC_IntegerLiteral
;
oC_Parameter
: '$' ( oC_SymbolicName | DecimalInteger ) ;
oC_PropertyExpression
: oC_Atom SP? oC_PropertyLookup ;
oC_PropertyKeyName
: oC_SchemaName ;
oC_IntegerLiteral
: DecimalInteger ;
DecimalInteger
: ZeroDigit
| ( NonZeroDigit ( Digit )* )
;
HexLetter
: ( 'A' | 'a' )
| ( 'B' | 'b' )
| ( 'C' | 'c' )
| ( 'D' | 'd' )
| ( 'E' | 'e' )
| ( 'F' | 'f' )
;
HexDigit
: Digit
| HexLetter
;
Digit
: ZeroDigit
| NonZeroDigit
;
NonZeroDigit
: NonZeroOctDigit
| '8'
| '9'
;
NonZeroOctDigit
: '1'
| '2'
| '3'
| '4'
| '5'
| '6'
| '7'
;
ZeroDigit
: '0' ;
oC_DoubleLiteral
: ExponentDecimalReal
| RegularDecimalReal
;
ExponentDecimalReal
: ( ( Digit )+ | ( ( Digit )+ '.' ( Digit )+ ) | ( '.' ( Digit )+ ) ) ( 'E' | 'e' ) '-'? ( Digit )+ ;
RegularDecimalReal
: ( Digit )* '.' ( Digit )+ ;
oC_SchemaName
: oC_SymbolicName ;
oC_SymbolicName
: UnescapedSymbolicName
| EscapedSymbolicName {if ($EscapedSymbolicName.text == "``") { notifyEmptyToken($EscapedSymbolicName); }}
| HexLetter
| kU_NonReservedKeywords
;
// example of BEGIN and END: TCKWith2.Scenario1
kU_NonReservedKeywords
: COMMENT
| ADD
| ALTER
| AS
| ATTACH
| BEGIN
| BY
| CALL
| CHECKPOINT
| COMMENT
| COMMIT
| CONTAINS
| COPY
| COUNT
| CYCLE
| DATABASE
| DECIMAL
| DELETE
| DETACH
| DROP
| EXPLAIN
| EXPORT
| EXTENSION
| FORCE
| GRAPH
| IF
| IS
| IMPORT
| INCREMENT
| KEY
| LOAD
| LOGICAL
| MATCH
| MAXVALUE
| MERGE
| MINVALUE
| NO
| NODE
| PROJECT
| READ
| REL
| RENAME
| RETURN
| ROLLBACK
| ROLE
| SEQUENCE
| SET
| START
| STRUCT
| L_SKIP
| LIMIT
| TRANSACTION
| TYPE
| USE
| UNINSTALL
| UPDATE
| WRITE
| FROM
| TO
| YIELD
| USER
| PASSWORD
| MAP
;
UnescapedSymbolicName
: IdentifierStart ( IdentifierPart )* ;
IdentifierStart
: ID_Start
| Pc
;
IdentifierPart
: ID_Continue
| Sc
;
EscapedSymbolicName
: ( '`' ( EscapedSymbolicName_0 )* '`' )+ ;
SP
: ( WHITESPACE )+ ;
WHITESPACE
: SPACE
| TAB
| LF
| VT
| FF
| CR
| FS
| GS
| RS
| US
| '\u1680'
| '\u180e'
| '\u2000'
| '\u2001'
| '\u2002'
| '\u2003'
| '\u2004'
| '\u2005'
| '\u2006'
| '\u2008'
| '\u2009'
| '\u200a'
| '\u2028'
| '\u2029'
| '\u205f'
| '\u3000'
| '\u00a0'
| '\u2007'
| '\u202f'
| CypherComment
;
CypherComment
: ( '/*' ( Comment_1 | ( '*' Comment_2 ) )* '*/' )
| ( '//' ( Comment_3 )* CR? ( LF | EOF ) )
;
oC_LeftArrowHead
: '<'
| '\u27e8'
| '\u3008'
| '\ufe64'
| '\uff1c'
;
oC_RightArrowHead
: '>'
| '\u27e9'
| '\u3009'
| '\ufe65'
| '\uff1e'
;
oC_Dash
: '-'
| '\u00ad'
| '\u2010'
| '\u2011'
| '\u2012'
| '\u2013'
| '\u2014'
| '\u2015'
| '\u2212'
| '\ufe58'
| '\ufe63'
| '\uff0d'
;
fragment FF : [\f] ;
fragment EscapedSymbolicName_0 : ~[`] ;
fragment RS : [\u001E] ;
fragment ID_Continue : [\p{ID_Continue}] ;
fragment Comment_1 : ~[*] ;
fragment StringLiteral_1 : ~['\\] ;
fragment Comment_3 : ~[\n\r] ;
fragment Comment_2 : ~[/] ;
fragment GS : [\u001D] ;
fragment FS : [\u001C] ;
fragment CR : [\r] ;
fragment Sc : [\p{Sc}] ;
fragment SPACE : [ ] ;
fragment Pc : [\p{Pc}] ;
fragment TAB : [\t] ;
fragment StringLiteral_0 : ~["\\] ;
fragment LF : [\n] ;
fragment VT : [\u000B] ;
fragment US : [\u001F] ;
fragment ID_Start : [\p{ID_Start}] ;
// This is used to capture unknown lexer input (e.g. !) to avoid parser exception.
Unknown : .;

View File

@@ -0,0 +1 @@
Neither `Cypher.g4` nor `keywords.txt` can be individually used to generate Ladybug's grammar. Rather, the files are combined to generate `scripts/antlr4/Cypher.g4`, which is immediately digestible.

View File

@@ -0,0 +1,115 @@
ACYCLIC
ANY
ADD
ALL
ALTER
AND
AS
ASC
ASCENDING
ATTACH
BEGIN
BY
CALL
CASE
CAST
CHECKPOINT
COLUMN
COMMENT
COMMIT
COMMIT_SKIP_CHECKPOINT
CONTAINS
COPY
COUNT
CREATE
CYCLE
DATABASE
DBTYPE
DEFAULT
DELETE
DESC
DESCENDING
DETACH
DISTINCT
DROP
ELSE
END
ENDS
EXISTS
EXPLAIN
EXPORT
EXTENSION
FALSE
FROM
FORCE
GLOB
GRAPH
GROUP
HEADERS
HINT
IMPORT
IF
IN
INCREMENT
INSTALL
IS
JOIN
KEY
LIMIT
LOAD
LOGICAL
MACRO
MATCH
MAXVALUE
MERGE
MINVALUE
MULTI_JOIN
NO
NODE
NOT
NONE
NULL
ON
ONLY
OPTIONAL
OR
ORDER
PRIMARY
PROFILE
PROJECT
READ
REL
RENAME
RETURN
ROLLBACK
ROLLBACK_SKIP_CHECKPOINT
SEQUENCE
SET
SHORTEST
START
STARTS
STRUCT
TABLE
THEN
TO
TRAIL
TRANSACTION
TRUE
TYPE
UNION
UNWIND
UNINSTALL
UPDATE
USE
WHEN
WHERE
WITH
WRITE
WSHORTEST
XOR
SINGLE
YIELD
USER
PASSWORD
ROLE
MAP

View File

@@ -0,0 +1,22 @@
add_subdirectory(bind)
add_subdirectory(bind_expression)
add_subdirectory(ddl)
add_subdirectory(expression)
add_subdirectory(query)
add_subdirectory(rewriter)
add_subdirectory(visitor)
add_library(lbug_binder
OBJECT
binder.cpp
binder_scope.cpp
bound_statement_result.cpp
bound_scan_source.cpp
bound_statement_rewriter.cpp
bound_statement_visitor.cpp
expression_binder.cpp
expression_visitor.cpp)
set(ALL_OBJECT_FILES
${ALL_OBJECT_FILES} $<TARGET_OBJECTS:lbug_binder>
PARENT_SCOPE)

Some files were not shown because too many files have changed in this diff Show More