jdormit/jeremy-website
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Rebuild site with Obelix site builder

Browse Source 
Squashed commit of the following:

commit b61428ef9441b6c6b68192483f20576576484c46
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 10:30:13 2020 -0500

    Remove draft post

commit b4e01335329091ea0618805515414498487e4569
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 10:29:17 2020 -0500

    Make RSS feed

commit f9c2956db1f44fea2c9f45d18aad189a950bad17
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 10:28:50 2020 -0500

    Update to Obelix 1.1.0 (no longer escapes HTML automatically)

commit 1d0b09fb904bee4d19af9eb96c8ec7945b4e4257
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 08:57:44 2020 -0500

    Don't draw <hr> on last blog summary

commit e97d3399614d04ae2f2364e18de60d8076d66ba4
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 08:53:00 2020 -0500

    Fix mobile layout

commit 4931b8153106702f33200f8098314a54151350d6
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 08:46:07 2020 -0500

    Dedupe tree branches correctly

commit 54708b956e1bc4722efbef5f01f96985891a702e
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 08:45:54 2020 -0500

    Add randomness to tree sketch

commit 81a68a8d1a92630394082f5218574b52aced56d1
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 08:45:29 2020 -0500

    Re-render sketch on window resize

commit eca26effffb5dbc1d881031a76d044608bd3fa69
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Thu Dec 10 08:44:46 2020 -0500

    Support stochastic l-system grammars

commit 6050d59408548c7d014da450d0e6737fe768f19f
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Mon Dec 7 21:07:00 2020 -0500

    Add TODO

commit d478bcc3f642dd58e93e8ef0542e19ea9acc6f1a
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Mon Dec 7 21:05:51 2020 -0500

    Don't sort l-system output; downsize to 6 iteration

commit f01f9135172e7b60480b64ede936e380384c0060
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Mon Dec 7 17:26:15 2020 -0500

    [WIP] Leave game plan comment

commit e191ea7b86e05098d2fcc2e6ea99c06e6ad493fb
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Mon Dec 7 17:17:44 2020 -0500

    [WIP] Don't draw lines multiple times

commit 7cbc9bd839743fa660962f036f34817c6deb05de
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Mon Dec 7 13:57:49 2020 -0500

    Batch size = 50

commit 571ea7728e2605456578be7f70e0c0806434b750
Author: Jeremy Dormitzer <jeremydormitzer@lola.com>
Date:   Mon Dec 7 13:55:19 2020 -0500

    Account for depth when rendering tree branches

commit 53469c671c0ecf92edaabd152b58a532dfb5f54f
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Mon Dec 7 08:45:50 2020 -0500

    Have the tree draw itself

commit 5b023911a346f72cb4fea56aaedebc371b0f7135
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Mon Dec 7 07:11:08 2020 -0500

    Calculate l-system once

commit 4b9daa8cd2e5790973e93d2ff3909a4791d3332b
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 11:38:15 2020 -0500

    Extract l-system->lines into a function

commit 7bcef9263aadcb00b0e8e045e23ccfca4ef22a0f
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 11:21:04 2020 -0500

    Tweak plant: more iterations, smaller segment length

commit 9b37f41af829d64ffd2513573eaf71ab4f6b1fe1
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 11:16:22 2020 -0500

    Move all calculations to update-state; use more natural l-system

commit 6496fa4b57487a97d037ecb5d357ccd0c1db13a1
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 11:16:10 2020 -0500

    Add l-system/rules macro

commit 17e1b0f867fe5ea6e188e67b90a5895d9d34cfef
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 11:15:39 2020 -0500

    Optimize l-system/step and allow strings as axioms

commit f7b35123c3f01e65449a2393645a5b4719c5a9e0
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 10:00:03 2020 -0500

    Implement test l-system

commit 2088e66fad63ca890c9c4b1c268d868d03d9a7f5
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 08:57:10 2020 -0500

    Use HSB color mode in tree sketch

commit bf7aac0636c1208a9226df7fce29fce633453cf2
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 08:57:01 2020 -0500

    Clean up main.cljs

commit 7c5a4cda9932d8ddcd29c5540fb326d444ce9241
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 08:56:35 2020 -0500

    Hide empty scroll bar on landing page

commit 92559812c1e7a6cc25a0f507f978028a2619fbf8
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 08:55:59 2020 -0500

    Initial l-system implementation

commit 2e8669c0434a0b26d7571b1cd7098ba12ce4323c
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Dec 6 06:51:31 2020 -0500

    Ignore cache files

commit 14c605639a7d3862dccb9ecc23f0c9fe23a34144
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Tue Dec 1 08:28:57 2020 -0500

    Lay out sketch on main page

commit c58155b13bcf285bdcc0d5ceb12ecda442631832
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Mon Nov 30 07:45:11 2020 -0500

    Fix makefile target name

commit 888ec38de1cfcfe184192a42313232a837cea49c
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Mon Nov 30 07:42:48 2020 -0500

    [WIP] Put a quil sketch on the homepage

commit 59145b883e77d2b32a483f44d93ebc47ea1ab493
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 29 10:57:18 2020 -0500

    Add Makefile with production build command

commit febd037d06a4d03efe5bad9c35ce67c169b61347
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 29 10:41:07 2020 -0500

    Remove :bundle-freq figwheel option

commit 4f15b81202c12270762c2145b224dae62bd544b8
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 29 09:35:22 2020 -0500

    Set up Figwheel build + obelix serve config

commit fece49200abd1348c7a2d9585c06878f9ce5ac0b
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sat Nov 21 15:10:11 2020 -0500

    Add cljs source

commit a0b2ed9c36b37bb83b35addcb39a791738704a92
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Thu Nov 19 21:35:23 2020 -0500

    Don't make images full-bleed

commit cb0b2f6e321616f1d4538bfaa86e266656356522
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Thu Nov 19 19:32:05 2020 -0500

    Match index page styling to existing site

commit 03f6b943ad2299789583ba35ad08a1686f703a92
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Thu Nov 19 08:26:27 2020 -0500

    Format the blog index page

commit 782c13fcdf1309806dd065af5383f92d5e1209cc
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Thu Nov 19 07:39:05 2020 -0500

    Sort blog posts by publish date on blog index

commit dd59865f6ef127604d0508302964fa130b834208
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Wed Nov 18 08:35:52 2020 -0500

    Update obelix-plugin-typography version

commit 682e69aff9ffb8aecfd3d2bc09b73e00a390d815
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Wed Nov 18 08:35:09 2020 -0500

    Style the header

commit 4ae624d42215c309f1e6c3b7fbbb0b43b83282b8
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Wed Nov 18 08:34:54 2020 -0500

    Add Google Fonts links

commit 8d3a3ffc3c32d6f40c37a7692fa0f87dd5a11f55
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 15 16:00:31 2020 -0500

    Add typography.js plugin

commit fdf743ad58d8a0978778f9577f766cae815e71b5
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 16:26:01 2020 -0500

    Sketch out site styling

commit 4a6b986d27a4aae364942da1571d2cb86a902963
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 16:25:53 2020 -0500

    Add titles

commit 3cd7ad4cef10a53ac94a90527667453c433344d5
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 16:25:23 2020 -0500

    Make blog headers h2s

commit 28106c2d37eefacca98160a09cf8572485e1f3f2
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 13:08:08 2020 -0500

    Fix markdown formatting

commit 2549d00cccb9c782b5dcc3b98eaf436e1e2e6d9b
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 13:07:57 2020 -0500

    Make header images full-bleed

commit 9256dc3c3e1f6c6db0772533e3b9003ebfaf240b
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 13:03:07 2020 -0500

    Add preliminary blog index

commit 3380b8c02a87abfac62edc09af79331b802ccd0d
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 13:02:57 2020 -0500

    Make sure index.md gets read as a page

commit cc0aecdf1d039602d69a5b2669826a99228b426f
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 13:02:40 2020 -0500

    Flesh out layout and add separate blog layout

commit e978957fe36a6bce93ade00b7e5f5c03e633d09a
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 13:02:07 2020 -0500

    Vendor in highlight.js

commit 60d3b7071ba128f026f9c9423c643fc21707cbdb
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sun Nov 8 13:01:22 2020 -0500

    Delete package.json and rely on globally-installed Obelix

commit 9d0cd7b46e52885012a1c94cca0a7c26811afef4
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Thu Nov 5 09:43:13 2020 -0500

    Port blog posts and index over to Obelix

commit 800f4a088bd72f999835280d3381e9307c32776f
Author: Jeremy Dormitzer <jeremy.dormitzer@gmail.com>
Date:   Sat Oct 10 15:06:19 2020 -0400

    Delete everything
This commit is contained in:
Jeremy Dormitzer 2020-12-10 10:42:04 -05:00
parent 64febcc68a
commit 444ecd51d0
200 changed files with 4464 additions and 3541 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.dir-locals.el
View File

@ -1,2 +1,2 @@
((nil . ((projectile-project-compilation-cmd . "raco pollen render -f -s src")
(projectile-project-run-cmd . "raco pollen start src"))))
((nil . ((projectile-project-run-cmd . "obelix serve")
(cider-default-cljs-repl . figwheel-main))))

15
.gitignore vendored
View File

@ -1,7 +1,8 @@
src/**/*.html
src/**/*.css
**/compiled/*
src/**/*.js
src/**/*.md
src/**/*.xml
out/
/node_modules/
/out/
/.log/
/cljs/.cpcache/
/src/cljs/
/target/
/.cpcache/
/.nrepl-port

5
Makefile Normal file
View File

@ -0,0 +1,5 @@
.PHONY: prod
prod:
clojure -m figwheel.main -O advanced -bo dev
obelix build

14
README.org
View File

@ -1,14 +0,0 @@
* Jeremy Dormitzer's personal website
This is my personal website. It's written using the [[https://docs.racket-lang.org/pollen/index.html][Pollen static site generator]].
* Running the development server
Make sure you have Pollen installed, then run:
#+BEGIN_SRC sh
raco pollen start
#+END_SRC
* Building the static site
#+BEGIN_SRC sh
raco pollen render -s src && raco pollen publish src out
#+END_SRC

53
cljs/jeremy_website/l_system.clj Normal file
View File

@ -0,0 +1,53 @@
(ns jeremy-website.l-system
(:require [clojure.core.match :refer [match]]
[clojure.string :as s]))
(defn parse-probability
[predecessor]
(let [prob-str (second (re-find #"\(((\d|\.)+)\)" predecessor))]
(when prob-str
(Double/parseDouble prob-str))))
(defn strip-probability
[predecessor]
(s/replace predecessor #"\(.*\)" ""))
(defn parse-rule
[predecessor successor]
(let [probability (parse-probability predecessor)
predecessor (strip-probability predecessor)]
{:probability (or probability 1.0)
:predecessor predecessor
:successor (s/split successor #"")}))
(defmacro rules
"Generates l-system rules.
Usage:
(rules
\"A(0.3)\" -> \"AB\"
\"A(0.7)\" -> \"BA\"
\"B\" -> \"A\")
which expands to:
[{:probability 0.3
:predecessor \"A\"
:successor [\"A\" \"B\"]}
{:probability 0.7
:predecessor \"A\"
:successor [\"B\" \"A\"]}
{:probability 1
:predecessor \"B\"
:successor [\"A\"]}]
"
[& rule-forms]
(let [rule-tuples (partition 3 3 (repeat nil) rule-forms)]
(vec (map (fn [tuple]
(match (vec tuple)
[(predecessor :guard #(string? %))
'->
(successor :guard #(string? %))]
(parse-rule predecessor successor)))
rule-tuples))))

43
cljs/jeremy_website/l_system.cljs Normal file
View File

@ -0,0 +1,43 @@
(ns jeremy-website.l-system)
;; TODO support context-sensitive grammers? This is where a production
;; rule can apply only if the symbol in question occurs between
;; specific other symbols
(defn rules-for-symbol
[symbol rules]
(filter #(= (:predecessor %) symbol) rules))
(defn choose-rule [random-fn rules]
(let [random (random-fn)
iter-fn (fn [[rule & rules] current-prob]
(cond
(nil? rule) nil
(< random (+ current-prob
(:probability rule))) rule
:else (recur rules (+ current-prob
(:probability rule)))))
selected (iter-fn rules 0)]
(or selected (first rules))))
(defn apply-rules
[rules random-fn symbol]
(let [rules-for-symbol (rules-for-symbol symbol rules)
selected-rule (choose-rule random-fn rules-for-symbol)]
(if selected-rule
(:successor selected-rule)
[symbol])))
(defn step
"Applies the l-system rules once"
[{:keys [rules state random-fn] :as l-system}]
(let [new-state (mapcat (partial apply-rules rules random-fn) state)]
(assoc l-system :state (vec new-state))))
(defn instantiate
"Instantiates a new L-system"
[{:keys [rules axiom random-fn] :or {random-fn rand}}]
{:rules rules
:axiom (vec axiom)
:state (vec axiom)
:random-fn random-fn})

22
cljs/jeremy_website/main.cljs Normal file
View File

@ -0,0 +1,22 @@
(ns jeremy-website.main
(:require [jeremy-website.sketches.tree :as tree]))
(enable-console-print!)
(defn fullscreen
[]
[js/document.body.clientWidth
(- js/window.innerHeight 120)])
(defn render-sketch
[]
(tree/sketch {:host "quil-sketch"
:size (fullscreen)}))
(render-sketch)
(.addEventListener
js/window
"resize"
(fn [_e]
(render-sketch)))

115
cljs/jeremy_website/sketches/tree.cljs Normal file
View File

@ -0,0 +1,115 @@
(ns jeremy-website.sketches.tree
(:require [jeremy-website.l-system :as l-system :include-macros true]
[quil.core :as q]
[quil.middleware :as m]))
(defn degrees->radians [deg]
(* deg (/ js/Math.PI 180)))
(defn l-system-lines
[{:keys [segment-length initial-angle initial-x initial-y l-system]}]
(vec
(:lines
(reduce
(fn [{:keys [x y angle angle-stack position-stack depth lines] :as state}
symbol]
(condp = symbol
"F" (let [new-x (+ x (* segment-length
(js/Math.cos (degrees->radians angle))))
new-y (+ y (* (- segment-length)
(js/Math.sin (degrees->radians angle))))
line {:x1 x :y1 y :x2 new-x :y2 new-y}]
(assoc state
:x new-x
:y new-y
:lines (conj lines (assoc line :depth depth))))
"+" (assoc state :angle (+ angle 25))
"-" (assoc state :angle (- angle 25))
"*" (assoc state :angle (+ angle 20))
"/" (assoc state :angle (- angle 20))
">" (assoc state :angle (+ angle 15))
"<" (assoc state :angle (- angle 15))
"[" (assoc state
:angle-stack (conj angle-stack angle)
:position-stack (conj position-stack [x y])
:depth (inc depth))
"]" (assoc state
:angle (peek angle-stack)
:angle-stack (pop angle-stack)
:x (first (peek position-stack))
:y (second (peek position-stack))
:position-stack (pop position-stack)
:depth (dec depth))
state))
{:angle initial-angle
:angle-stack [initial-angle]
:x initial-x
:y initial-y
:position-stack [[initial-x initial-y]]
:depth 0
:lines []}
(:state l-system)))))
(defn setup []
;; Hue goes from 0-360, saturation/brightness from 0-100
(q/color-mode :hsb 360 100 100)
(q/frame-rate 60)
(q/random-seed (or (-> (js/URLSearchParams. js/window.location.search)
(.get "seed"))
(.now js/Date)))
(let [l-system (-> (l-system/instantiate {:axiom "X"
:rules (l-system/rules
"X(0.4)" -> "F+[[X]-X]-F[-FX]+X"
"X(0.4)" -> "F*[[X]/X]/F[/FX]*X"
"X(0.2)" -> "F>[[X]<X]<F[<FX]>X"
"F" -> "FF")
:random-fn (partial q/random 1)})
((apply comp (repeat 6 l-system/step))))
lines (l-system-lines {:initial-angle (- 90 25)
:segment-length 6
:initial-x (* (q/width) 0.1)
:initial-y (* (q/height) 0.95)
:l-system l-system})]
{:to-draw lines
:drawing []}))
(defn update-state [{:keys [to-draw drawing] :as state}]
(let [batch-size 10]
(if (> (count to-draw) 0)
(assoc state
:to-draw (vec (drop batch-size to-draw))
:drawing (vec (concat drawing (take batch-size to-draw))))
state)))
(defn dedupe-lines [lines]
(:lines
(reduce (fn [{:keys [lines already-seen] :as state} line]
(if-not (contains? already-seen ((juxt :x1 :y1 :x2 :y2) line))
(assoc state
:already-seen (conj already-seen ((juxt :x1 :y1 :x2 :y2) line))
:lines (conj lines line))
state))
{:lines []
:already-seen #{}}
lines)))
(defn draw [{:keys [to-draw drawing]}]
(q/background 0 0 100)
(q/fill 0 0 0)
(let [lines (dedupe-lines drawing)]
(doseq [line lines]
(q/stroke-weight (- 1.75 (* 0.1 (:depth line))))
(q/stroke 240 0 (+ 65 (* 2 (:depth line))))
(apply q/line ((juxt :x1 :y1 :x2 :y2) line))))
(when (= (count to-draw) 0)
(q/no-loop)))
(defn sketch
[{:keys [host size]}]
(q/sketch
:host host
:middleware [m/fun-mode]
:size size
:setup setup
:update update-state
:draw draw))

7
deps.edn Normal file
View File

@ -0,0 +1,7 @@
{:deps {org.clojure/clojurescript {:mvn/version "1.10.764"}
org.clojure/core.match {:mvn/version "1.0.0"}
com.bhauman/figwheel-main {:mvn/version "0.2.12"}
org.clojure/tools.cli {:mvn/version "1.0.194"}
quil {:mvn/version "3.1.0"}}
:paths ["cljs" "out"]
:aliases {:fig {:main-opts ["-m" "figwheel.main"]}}}

5
dev.cljs.edn Normal file
View File

@ -0,0 +1,5 @@
^{:auto-bundle :webpack
:open-url false}
{:main jeremy-website.main
:output-dir "src/cljs"
:asset-path "cljs"}

19
helpers.js Normal file
View File

@ -0,0 +1,19 @@
const moment = require("moment-timezone");
function formatDate(date) {
return moment(date).format("MMMM D, YYYY");
}
function rfc822Date(date) {
return moment(date).tz("America/New_York").format("ddd, D MMM YYYY HH:mm:ss z");
}
function now() {
return new Date();
}
module.exports = {
formatDate,
rfc822Date,
now
};

9
obelix.edn Normal file
View File

@ -0,0 +1,9 @@
{:src "src"
:out "out"
:handlebarsHelpers "helpers.js"
:plugins
{:obelix-plugin-typography
{:fileName "css/typography.css"
:theme "typography-theme-moraga"}
:obelix-plugin-excerpt
{:pruneLength 500}}}

1358
package-lock.json generated Normal file
View File

File diff suppressed because it is too large Load Diff

20
package.json Normal file
View File

@ -0,0 +1,20 @@
{
"name": "jeremy-website",
"version": "1.0.0",
"description": "My personal website",
"dependencies": {
"moment": "^2.29.1",
"moment-timezone": "^0.5.32",
"obelix-plugin-excerpt": "^1.0.0",
"obelix-plugin-typography": "^1.0.2",
"typography-theme-moraga": "^0.16.19",
"voronoi": "^1.0.0",
"webpack": "^5.6.0",
"webpack-cli": "^4.2.0"
},
"repository": {
"type": "git",
"url": "git@git.jeremydormitzer.com:jdormit/jeremy-website"
},
"author": "Jeremy Dormitzer <jeremy.dormitzer@gmail.com>"
}

58
src/blog/a-dsl-for-music.html.pm → src/blog/a-dsl-for-music.md
View File

@ -1,23 +1,23 @@
#lang pollen
◊(define-meta title "A DSL for Music")
◊(define-meta published "2018-08-05")
◊heading{Haskell School of Music}
---
title: A DSL for music
author: Jeremy Dormitzer
published: 2018-08-05
---
## Haskell School of Music
I recently discovered Haskell School of Music. It’s a book about algorithmic music, which is awesome because: a) I’ve been obsessed with procedural generation for years and b) I like music as much as I like programming. So you can imagine my excitement when I discovered that someone had written a textbook combining my favorite areas of study.
Haskell School of Music is aimed at intermediate-level CS students, so it covers a lot of the basics of functional programming. It aims to be an introduction to the Haskell programming language while also thoroughly examining computer music. It starts simply by defining the data structures that represent music, and progresses to functional programming concepts, procedurally generating music, and doing signal processing and MIDI interfacing to actually play the songs.
I like Haskell, but I want to write music in Clojure. Why? First of all, because it’s the One True Language (it’s fine if you disagree with me – your opinion is valid even if it’s objectively incorrect). But more importantly, Clojure excels as an environment for writing ◊link[#:href "https://en.wikipedia.org/wiki/Domain-specific_language"]{domain-specific languages} (DSLs). And as it turns out, writing algorithmic music using a DSL is a major win. Not only are DSLs expressive enough to portray creative expression, but DSLs written in Lisps are inherently extensible – whereas Haskell’s static typing adds barriers to extensibility. There’s also an excellent music synthesis library for Clojure, ◊link[#:href "https://overtone.github.io/"]{Overtone}, that I want to be able to take advantage of.
I like Haskell, but I want to write music in Clojure. Why? First of all, because it’s the One True Language (it’s fine if you disagree with me – your opinion is valid even if it’s objectively incorrect). But more importantly, Clojure excels as an environment for writing [domain-specific languages](https://en.wikipedia.org/wiki/Domain-specific_language) (DSLs). And as it turns out, writing algorithmic music using a DSL is a major win. Not only are DSLs expressive enough to portray creative expression, but DSLs written in Lisps are inherently extensible – whereas Haskell’s static typing adds barriers to extensibility. There’s also an excellent music synthesis library for Clojure, [Overtone](https://overtone.github.io/), that I want to be able to take advantage of.
Before we can explore what a DSL for music would look like, we need to understand how HSoM represents music as data.
◊heading{Music as data}
## Music as data
HSoM breaks music down into its component pieces. It represents music using Haskell data structures:
◊codeblock[#:lang "haskell"]{
```haskell
type Octave = Int
data PitchClass = Cff | Cf | C | Dff -- ...etc, all the way to Bss
type Pitch = (PitchClass, Octave)
@ -33,35 +33,35 @@ data Music a =
| Music a :+: Music a
| Music a :=: Music a
| Modify Control (Music a)
}
```
Many of these type declarations are straightfoward, but a couple bear further discussion. A ◊code{PitchClass} is an ◊link[#:href "https://wiki.haskell.org/Algebraic_data_type"]{algebraic data type} representing all of the pitches: C#, Ab, F, and so on. By pairing a pitch class with an octave, we get a ◊code{Pitch}, which represents a specific note (for instance, middle C would be ◊code{(C, 4)}. A ◊code{Primitive} is a basic music building block, either a note or a rest. Note that it is ◊link[#:href "https://wiki.haskell.org/Polymorphism"]{polymorphic}: this is so that we can define types like ◊code{Note Duration Pitch} but also types like ◊code{Note Duration (Pitch, Loudness)} so that we can attach additional data to a primitive if we need to. A control represents the concept of making a modification to some music by changing the tempo, transposing it, or otherwise changing the output while keeping the underlying notes the same.
Many of these type declarations are straightfoward, but a couple bear further discussion. A `PitchClass` is an [algebraic data type](https://wiki.haskell.org/Algebraic_data_type) representing all of the pitches: C#, Ab, F, and so on. By pairing a pitch class with an octave, we get a `Pitch`, which represents a specific note (for instance, middle C would be `(C, 4)`. A `Primitive` is a basic music building block, either a note or a rest. Note that it is [polymorphic](https://wiki.haskell.org/Polymorphism): this is so that we can define types like `Note Duration Pitch` but also types like `Note Duration (Pitch, Loudness)` so that we can attach additional data to a primitive if we need to. A control represents the concept of making a modification to some music by changing the tempo, transposing it, or otherwise changing the output while keeping the underlying notes the same.
The ◊code{Music} type is where things get really interesting. It’s an algebraic data type representing the concept of music in general. In fact, it’s powerful enough to fully represent any piece of music, from Hozier to Bach. A ◊code{Music} value is one of four possible types: a ◊code{Prim}, which is either a note or a rest; a ◊code{Modify}, which takes another ◊code{Music} as an argument and modifies it in some way; the ◊code{:+:} ◊link[#:href "https://downloads.haskell.org/~ghc/7.2.1/docs/html/users_guide/data-type-extensions.html"]{infix constructor}, which represents two separate ◊code{Music} values played sequentially; and the ◊code{:=:} infix constructor, which represents two separate ◊code{Music} values played simultaneously.
The `Music` type is where things get really interesting. It’s an algebraic data type representing the concept of music in general. In fact, it’s powerful enough to fully represent any piece of music, from Hozier to Bach. A `Music` value is one of four possible types: a `Prim`, which is either a note or a rest; a `Modify`, which takes another `Music` as an argument and modifies it in some way; the `:+:` [infix constructor](https://downloads.haskell.org/~ghc/7.2.1/docs/html/users_guide/data-type-extensions.html), which represents two separate `Music` values played sequentially; and the `:=:` infix constructor, which represents two separate `Music` values played simultaneously.
The ◊code{Music} type has some important properties. First, it’s polymorphic for the same reason that the ◊code{Primitive} type is. This allows us to attach any type of data we want to music primitives, letting us express any musical concept (volume, gain, you name it).
The `Music` type has some important properties. First, it’s polymorphic for the same reason that the `Primitive` type is. This allows us to attach any type of data we want to music primitives, letting us express any musical concept (volume, gain, you name it).
Second, three out of its four constructors are recursive – they take other ◊code{Music} values as arguments. This is the key that makes the data model so powerful. It allows you to model arbitrary configurations of notes, e.g. ◊code{Note 1/4 (C 4) :=: Note 1/4 (E 4) :=: Note 1/4 (G 4)} is a C major triad, and that expression evaulates to a ◊code{Music} value that can itself be passed to ◊code{Modify}, ◊code{:+:}, or ◊code{:=:} to weave it into a larger piece of music.
Second, three out of its four constructors are recursive – they take other `Music` values as arguments. This is the key that makes the data model so powerful. It allows you to model arbitrary configurations of notes, e.g. `Note 1/4 (C 4) :=: Note 1/4 (E 4) :=: Note 1/4 (G 4)` is a C major triad, and that expression evaulates to a `Music` value that can itself be passed to `Modify`, `:+:`, or `:=:` to weave it into a larger piece of music.
The result is an extraordinarily concise definition that still manages to encompass all possible pieces of music. Using these data structures, we can describe any song we can imagine.
But as powerful as this data type is, I wouldn’t call it a domain-specific language. The static type system makes it inflexible: how would you combine a ◊code{Note 1/8 ((C 4) 8)}, representing a note with pitch and loudness, with a ◊code{Note 1/8 (E 4)}, representing a note with just a pitch? Sure, you could write a function to convert from one to the other, but at that point you’ve lost elegance and flexibility.
But as powerful as this data type is, I wouldn’t call it a domain-specific language. The static type system makes it inflexible: how would you combine a `Note 1/8 ((C 4) 8)`, representing a note with pitch and loudness, with a `Note 1/8 (E 4)`, representing a note with just a pitch? Sure, you could write a function to convert from one to the other, but at that point you’ve lost elegance and flexibility.
Here’s where Clojure comes in.
◊heading{A DSL for music with Clojure}
## A DSL for music with Clojure
What would a domain-specific language for music look like in Clojure? I found inspiration in the HTML templating library ◊link[#:href "https://github.com/weavejester/hiccup"]{Hiccup}. Hiccup represents HTML documents (a graph of complex nested nodes, just like music values) using Clojure vectors, like so:
What would a domain-specific language for music look like in Clojure? I found inspiration in the HTML templating library [Hiccup](https://github.com/weavejester/hiccup). Hiccup represents HTML documents (a graph of complex nested nodes, just like music values) using Clojure vectors, like so:
◊codeblock[#:lang "clojure"]{
```clojure
[:div {:class "foo"} [:p "foo"]]
}
```
The Hiccup vectors are actually a DSL that can describe arbitrary HTML markup. Anything that can be expressed in HTML can be expressed using Hiccup. It straddles the line between data and code – the vector is flexible and expressive enough to represent any web page, but can be manipulated using standard Clojure library functions.
If we apply this idea to the data structure from HSoM, we end up with something like this:
◊codeblock[#:lang "clojure"]{
```clojure
;; notes and rests are maps
(def eighth-note-c
{:duration 1/8
@ -99,36 +99,34 @@ If we apply this idea to the data structure from HSoM, we end up with something
:pitch [C 4]}
{:duration 1/4
:pitch [E 4]}]]]
}
```
At first glance, this looks the same as the Haskell data types from HSoM. Both representations represent notes with pitch and duration; both use the ◊code{:modify}, ◊code{:=} and ◊code{:+} operators to compose music; both support recursive composition of any depth.
At first glance, this looks the same as the Haskell data types from HSoM. Both representations represent notes with pitch and duration; both use the `:modify`, `:=` and `:+` operators to compose music; both support recursive composition of any depth.
But the Clojure version is actually more expressive and flexible than the Haskell equivalent. A note can have any metadata we want attached:
◊codeblock[#:lang "clojure"]{
```clojure
{:duration 1/4
:pitch [:C 4]
:loudness 6}
}
```
Our DSL has no problem composing notes with differing metadata:
◊codeblock[#:lang "clojure"]{
```clojure
[:+
{:duration 1/4
:pitch [:C 4]
:loudness 6}
{:duration 1/8
:pitch [:Eb 4]
:staccato true}]
}
```
Furthermore, because Clojure is dynamically typed and supports ◊link[#:href "https://en.wikipedia.org/wiki/Duck_typing"]{duck typing} via map keywords, we can write functions that operate on all notes and music values, even those with unexpected metadata.
Furthermore, because Clojure is dynamically typed and supports [duck typing](https://en.wikipedia.org/wiki/Duck_typing) via map keywords, we can write functions that operate on all notes and music values, even those with unexpected metadata.
Like the Hiccup vectors, our music vectors blur the boundary between a DSL and a data structure. The vectors are expressive enough to represent any musical concept, but can still be passed around and operated on by normal Clojure functions. As an added advantage, the vectors look similar enough to the HSoM data structures that I can easily follow along with the textbook using Clojure and Overtone.
◊heading{What's next}
## What's next
So I have a way to represent music in Clojure now. What’s next? Haskell School of Music ships with a library called Euterpea that knows how to turn the Music data structure into actual sound. So the next step for me is probably porting something like that to Clojure. I’m hoping to offload most of that work to Overtone. After that, I’ll explore algorithmic composition using the techniques outlined in HSoM. Stay tuned!
◊(define-meta tags '("clojure" "music" "procedural generation"))

27
src/blog/activitypub-good-enough-for-jazz.html.pm
View File

@ -1,27 +0,0 @@
#lang pollen
◊(define-meta title "ActivityPub: Good enough for jazz")
◊(define-meta published "2019-01-07")
◊heading-image[#:src "/images/activitypub.png"]
◊link[#:href "https://pleroma.site/users/kaniini"]{Kaniini}, one of the lead developers of Pleroma, recently published a blog post called ◊link[#:href "https://blog.dereferenced.org/activitypub-the-worse-is-better-approach-to-federated-social-networking"]{ActivityPub: The “Worse is Better” Approach to Federated Social Networking}. It’s a critique of the security and safety of the ◊link[#:href "https://jeremydormitzer.com/blog/what-is-activitypub-and-how-will-it-change-the-internet/"]{ActivityPub protocol}. They make some good points:
◊ul{
◊li{ActivityPub doesn’t support fine-grained access control checks, e.g. I want someone to be able to see my posts but not respond to them}
◊li{Instances you’ve banned can still see threads from your instance in some ActivityPub implementations, because someone from a third instance replies to the thread and that reply reaches the banned instance}
}
The post also generated an ◊link[#:href "https://playvicious.social/@Are0h/101372851868909058"]{interesting Fediverse thread} discussing the tradeoffs between proliferating the existing protocol versus making changes to it, and whether it would be possible to improve the protocol without breaking backward compatibility. It’s worth a read.
Here’s the thing: ActivityPub is a protocol, and protocols are only valuable as long as there is software out there actually using the protocol. At the end of the day, that’s the most important measure of success. Don’t get me wrong – protocols need to do the job they set out to do well. But at some point, the protocol works well enough that it becomes more important to foster adoption than to continue improving. I believe that ActivityPub has reached that point.
Now, I’m not suggesting that we stop development on the protocol. But future improvements to it should be iterative, building on the existing specification, and backward compatible whenever possible. For example, by all means let’s come up with a better access control model for ActivityPub – but we should also come up with a compatibility layer that assumes some default set of access capabilities for implementations that haven’t upgraded. This lets us move forward without leaving the protocol’s participants behind, preserving ActivityPub’s value.
We are in good company here. This model is exactly how HTTP became the protocol that powers the internet. If you have the time, check out this ◊link[#:href "https://hpbn.co/brief-history-of-http/"]{excellent (brief) history} of the HTTP protocol. Here are the highlights: Tim Berners-Lee came up with HTTP 0.9, which was an extremely simple protocol that allowed clients to request a resource and receive a response. HTTP 1.0 added headers and a variety of other features. HTTP 1.1 added performance optimizations and fixed ambiguities in the 1.0 specification.
Critically, all of these versions of HTTP were similar enough that a server that supported HTTP 1.1 could trivially also support HTTP 1.0 and 0.9 (because 0.9 is actually a subset of 1.1). In fact, the Apache and Nginx web servers, which power most websites on the internet, still support HTTP 0.9! By designing and iterating on HTTP in a way that preserved backward compatibility, the early web pioneers were able to build a robust, performant, secure protocol while still encouraging global adoption.
If we want the Fediverse to be just as robust, performant, secure, and globally adopted, we should take the same approach.
◊(define-meta tags '("activitypub" "fediverse"))

23
src/blog/activitypub-good-enough-for-jazz.md Normal file
View File

@ -0,0 +1,23 @@
---
title: "ActivityPub: Good enough for jazz"
author: Jeremy Dormitzer
published: 2019-01-07
---
<img alt="activitypub" src="/images/activitypub.png" />
[Kaniini](https://pleroma.site/users/kaniini), one of the lead developers of Pleroma, recently published a blog post called [ActivityPub: The “Worse is Better” Approach to Federated Social Networking](https://blog.dereferenced.org/activitypub-the-worse-is-better-approach-to-federated-social-networking). It’s a critique of the security and safety of the [ActivityPub protocol](https://jeremydormitzer.com/blog/what-is-activitypub-and-how-will-it-change-the-internet/). They make some good points:
- ActivityPub doesn’t support fine-grained access control checks, e.g. I want someone to be able to see my posts but not respond to them
- Instances you’ve banned can still see threads from your instance in some ActivityPub implementations, because someone from a third instance replies to the thread and that reply reaches the banned instance
The post also generated an [interesting Fediverse thread](https://playvicious.social/@Are0h/101372851868909058) discussing the tradeoffs between proliferating the existing protocol versus making changes to it, and whether it would be possible to improve the protocol without breaking backward compatibility. It’s worth a read.
Here’s the thing: ActivityPub is a protocol, and protocols are only valuable as long as there is software out there actually using the protocol. At the end of the day, that’s the most important measure of success. Don’t get me wrong – protocols need to do the job they set out to do well. But at some point, the protocol works well enough that it becomes more important to foster adoption than to continue improving. I believe that ActivityPub has reached that point.
Now, I’m not suggesting that we stop development on the protocol. But future improvements to it should be iterative, building on the existing specification, and backward compatible whenever possible. For example, by all means let’s come up with a better access control model for ActivityPub – but we should also come up with a compatibility layer that assumes some default set of access capabilities for implementations that haven’t upgraded. This lets us move forward without leaving the protocol’s participants behind, preserving ActivityPub’s value.
We are in good company here. This model is exactly how HTTP became the protocol that powers the internet. If you have the time, check out this [excellent (brief) history](https://hpbn.co/brief-history-of-http/) of the HTTP protocol. Here are the highlights: Tim Berners-Lee came up with HTTP 0.9, which was an extremely simple protocol that allowed clients to request a resource and receive a response. HTTP 1.0 added headers and a variety of other features. HTTP 1.1 added performance optimizations and fixed ambiguities in the 1.0 specification.
Critically, all of these versions of HTTP were similar enough that a server that supported HTTP 1.1 could trivially also support HTTP 1.0 and 0.9 (because 0.9 is actually a subset of 1.1). In fact, the Apache and Nginx web servers, which power most websites on the internet, still support HTTP 0.9! By designing and iterating on HTTP in a way that preserved backward compatibility, the early web pioneers were able to build a robust, performant, secure protocol while still encouraging global adoption.
If we want the Fediverse to be just as robust, performant, secure, and globally adopted, we should take the same approach.

36
src/blog/annnouncing-pterotype.html.pm
View File

@ -1,36 +0,0 @@
#lang pollen
◊(define-meta title "Announcing Pterotype")
◊(define-meta published "2018-11-15")
◊header-image[#:src "/images/pterotype.png"]
In ◊link[#:href "https://jeremydormitzer.com/blog/what-is-activitypub.html"]{my last post}, I wrote about an emerging web standard called ActivityPub that lets web services interoperate and form a federated, open social network. I made an argument about how important this new standard is – how it tears down walled gardens, discourages monopolies and centralization, and encourages user freedom.
I genuinely believe what I wrote, too. And so, to put my money where my mouth is, I’m excited to announce ◊link[#:href "https://getpterotype.com/"]{Pterotype}! It’s a WordPress plugin that gives your blog an ActivityPub feed so that it can take advantage of all the benefits ActivityPub has to offer.
◊heading{Why WordPress?}
My mission is to open up the entire internet. I want every website, every social network, and every blog to be a part of the Fediverse. And WordPress ◊link[#:href "https://w3techs.com/technologies/overview/content_management/all"]{runs literally 30% of the internet}. It’s not my favorite piece of software, and I certainly never expected to write any PHP, but the fact is that writing a WordPress plugin is the highest-impact way to grow the Fediverse the fastest.
◊heading{So wait, what does this actually do?}
Great question, glad you asked. Pterotype makes your blog look like a Mastodon/Pleroma/whatever account to users on those platforms. So, if you install Pterotype on your blog, Mastodon users will be able to search for ◊code{blog@yourawesomesite.com} in Mastodon and see your blog as if it was a Mastodon user. If they follow your blog within Mastodon (or Pleroma, or…), your new posts will show up in their home feed. This is what I meant in my last post about ActivityPub making sites first-class citizens in social networks – you don’t need a Mastodon account to make this work, and your content will show up in any service that implements ActivityPub without you needing an account on those platforms either.
Here’s what this blog looks like from Mastodon:
◊image[#:src "/images/jeremy-mastodon.png"]
The plugin also syncs up comments between WordPress and the Fediverse. Replies from Mastodon et. al on your posts will show as WordPress comments, and comments from WordPress will show up as replies in the Fediverse. This is what I meant about tearing down walled gardens: people can comment on your blog posts using the platform of their choice, instead of being limited by the platform hosting the content.
◊heading{Sounds amazing! Can I use it now?}
Yes, with caveats. Pterotype is in early beta. The core features are in there – your blog will get a Fediverse profile, posts will federate, and comments will sync up – but it’s a pretty fiddly (and sometimes buggy) experience at the moment. If you do want to try it out, the plugin is in the ◊link[#:href "https://wordpress.org/plugins/pterotype/"]{plugin repository}. If you install it on your blog, please consider ◊link[#:href "https://getpterotype.com/beta"]{signing up for the beta program} as well – it’s how I’m collecting feedback and bug reports so I can make the plugin the best that it can be.
If you’d rather just follow my progress and dive in when it’s finished, that’s fine too! I made my development roadmap ◊link[#:href "https://getpterotype.com/roadmap"]{publicly available}, and the plugin itself is open-source ◊link[#:href "https://github.com/pterotype-project/pterotype"]{on GitHub}. I’m currently doing a major refactor, pulling out all of the ActivityPub-related logic ◊link[#:href "https://github.com/pterotype-project/activitypub-php"]{into its own library} – once that’s done, it’ll be back to business as usual adding features and stability to Pterotype.
If you’ve read this far, and this project resonates with you, then you might be interested in ◊link[#:href "https://www.patreon.com/pterotype"]{becoming a sponsor on Patreon}. Pterotype is free and open-source, so this is its only source of funding. For moment-to-moment updates, you can ◊link[#:href "https://mastodon.technology/@jdormit"]{follow me on Mastodon}.
See you on the Fediverse!
◊(define-meta tags '("activitypub" "fediverse" "pterotype" "wordpress"))

34
src/blog/announcing-pterotype.md Normal file
View File

@ -0,0 +1,34 @@
---
title: Announcing Pterotype
author: Jeremy Dormitzer
published: 2018-11-15
---
<img alt="a cute pterodactyl" src="/images/pterotype.png" />
In [my last post](https://jeremydormitzer.com/blog/what-is-activitypub.html), I wrote about an emerging web standard called ActivityPub that lets web services interoperate and form a federated, open social network. I made an argument about how important this new standard is – how it tears down walled gardens, discourages monopolies and centralization, and encourages user freedom.
I genuinely believe what I wrote, too. And so, to put my money where my mouth is, I’m excited to announce [Pterotype](https://getpterotype.com/)! It’s a WordPress plugin that gives your blog an ActivityPub feed so that it can take advantage of all the benefits ActivityPub has to offer.
## Why WordPress?
My mission is to open up the entire internet. I want every website, every social network, and every blog to be a part of the Fediverse. And WordPress [runs literally 30% of the internet](https://w3techs.com/technologies/overview/content_management/all). It’s not my favorite piece of software, and I certainly never expected to write any PHP, but the fact is that writing a WordPress plugin is the highest-impact way to grow the Fediverse the fastest.
## So wait, what does this actually do?
Great question, glad you asked. Pterotype makes your blog look like a Mastodon/Pleroma/whatever account to users on those platforms. So, if you install Pterotype on your blog, Mastodon users will be able to search for `blog@yourawesomesite.com` in Mastodon and see your blog as if it was a Mastodon user. If they follow your blog within Mastodon (or Pleroma, or…), your new posts will show up in their home feed. This is what I meant in my last post about ActivityPub making sites first-class citizens in social networks – you don’t need a Mastodon account to make this work, and your content will show up in any service that implements ActivityPub without you needing an account on those platforms either.
Here’s what this blog looks like from Mastodon:
![my website on Mastodon](/images/jeremy-mastodon.png)
The plugin also syncs up comments between WordPress and the Fediverse. Replies from Mastodon et. al on your posts will show as WordPress comments, and comments from WordPress will show up as replies in the Fediverse. This is what I meant about tearing down walled gardens: people can comment on your blog posts using the platform of their choice, instead of being limited by the platform hosting the content.
## Sounds amazing! Can I use it now?
Yes, with caveats. Pterotype is in early beta. The core features are in there – your blog will get a Fediverse profile, posts will federate, and comments will sync up – but it’s a pretty fiddly (and sometimes buggy) experience at the moment. If you do want to try it out, the plugin is in the [plugin repository](https://wordpress.org/plugins/pterotype/). If you install it on your blog, please consider [signing up for the beta program](https://getpterotype.com/beta) as well – it’s how I’m collecting feedback and bug reports so I can make the plugin the best that it can be.
If you’d rather just follow my progress and dive in when it’s finished, that’s fine too! I made my development roadmap [publicly available](https://getpterotype.com/roadmap), and the plugin itself is open-source [on GitHub](https://github.com/pterotype-project/pterotype). I’m currently doing a major refactor, pulling out all of the ActivityPub-related logic [into its own library](https://github.com/pterotype-project/activitypub-php) – once that’s done, it’ll be back to business as usual adding features and stability to Pterotype.
If you’ve read this far, and this project resonates with you, then you might be interested in [becoming a sponsor on Patreon](https://www.patreon.com/pterotype). Pterotype is free and open-source, so this is its only source of funding. For moment-to-moment updates, you can [follow me on Mastodon](https://mastodon.technology/@jdormit).
See you on the Fediverse!

23
src/blog/feed.xml.hbs Normal file
View File

@ -0,0 +1,23 @@
<rss version="2.0">
<channel>
<title>Jeremy Dormitzer's blog</title>
<link>https://jeremydormitzer.com/blog</link>
<description>Programming and general geekiness from Jeremy Dormitzer</description>
<language>en-us</language>
<copyright>© Jeremy Dormitzer 2020</copyright>
<ttl>60</ttl>
<lastBuildDate>{{ rfc822Date (now) }}</lastBuildDate>
<pubDate>{{ rfc822Date (now) }}</pubDate>
<docs>https://validator.w3.org/feed/docs/rss2.html</docs>
<generator>https://github.com/obelix-site-builder/obelix</generator>
{{#each (reverse (sort pages key="published"))}}
<item>
<title>{{ title }}</title>
<link>https://jeremydormitzer.com{{ url }}</link>
<guid>https://jeremydormitzer.com{{ url }}</guid>
<pubDate>{{rfc822Date published}}</pubDate>
<description>{{ content }}</description>
</item>
{{/each}}
</channel>
</rss>

56
src/blog/feed.xml.pp
View File

@ -1,56 +0,0 @@
#lang pollen
◊(require gregor
pollen/core
pollen/pagetree
pollen/file
pollen/template
racket/path
racket/string)
◊(define here-path (select-from-metas 'here-path metas))
◊(define ptree (get-pagetree (simplify-path (build-path (path-only here-path) 'up "index.ptree"))))
◊(current-pagetree ptree)
◊(define (rfc822 dt) (~t dt "E, dd MMM yyyy HH:mm:ss Z"))
◊(define (render-item item)
(let* ((src (get-source (path->string (path->complete-path (symbol->string item)))))
(link (format "https://jeremydormitzer.com/blog/~a" item)))
(format
"<item>
<title>~a</title>
<link>~a</link>
<guid>~a</guid>
<description>~a</description>
<pubDate>~a</pubDate>
</item>"
(select 'h1 src)
link
link
(->html (->html (get-doc src)))
(rfc822 (with-timezone
(at-midnight
(parse-date (select-from-metas 'published src) "yyyy-MM-dd"))
"America/New_York")))))
◊(define (render-items items)
(string-join (map render-item items) "\n"))
◊(define today (rfc822 (now/moment)))
<rss version="2.0">
<channel>
<title>Jeremy Dormitzer's blog</title>
<link>https://jeremydormitzer.com/blog</link>
<description>Programming and general geekiness from Jeremy Dormitzer.</description>
<language>en-us</language>
<copyright>© Jeremy Dormitzer 2019</copyright>
<ttl>60</ttl>
<lastBuildDate>◊|today|</lastBuildDate>
<pubDate>◊|today|</pubDate>
<docs>https://validator.w3.org/feed/docs/rss2.html</docs>
◊(render-items (sort (children 'blog) date>? #:key post-published-date))
</channel>
</rss>

10
src/blog/index.html.hbs Normal file
View File

@ -0,0 +1,10 @@
---
title: Jeremy Dormitzer’s blog
---
{{#each (reverse (sort pages key="published"))}}
<h2>{{ title }}</h2>
<p>Posted on {{formatDate published}}</p>
<p>{{{ excerpt }}}</p>
<p><a href="{{ url }}">Read more...</a></p>
{{#unless @last}}<hr>{{/unless}}
{{/each}}

32
src/blog/index.html.pm
View File

@ -1,32 +0,0 @@
#lang pollen
◊(require pollen/core
pollen/pagetree
pollen/file
gregor
racket/path
racket/list)
◊(define here-path (select-from-metas 'here-path metas))
◊(define ptree (get-pagetree (build-path
(path-only (string->path here-path))
'up
"index.ptree")))
◊(current-pagetree ptree)
◊(define (get-posts)
(sort (children 'blog)
date>?
#:key post-published-date))
◊(define (render-post post)
(let ((src (get-source (path->string (path->complete-path (symbol->string post))))))
(div (title (select 'h1 src))
(published-date (select-from-metas 'published src))
(excerpt post)
(link "Read more..." #:href (symbol->string post)))))
◊(let ((rendered-posts (add-between (map render-post (get-posts)) (divider))))
`(div ,@rendered-posts))
◊(define-meta browser-title "Jeremy Dormitzer's blog")

33
src/blog/layout.html.hbs Normal file
View File

@ -0,0 +1,33 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="/css/typography.css">
<link rel="stylesheet" href="/css/styles.css">
<title>{{ title }}</title>
</head>
<body>
<header>
<div class="header-container">
<nav>
<span>Jeremy Dormitzer</span>
<a href="/">Home</a>
<a href="/blog">Blog</a>
</nav>
</div>
</header>
<main class="container">
<h1>{{ title }}</h1>
{{{ content }}}
</main>
<footer>
© Jeremy Dormitzer 2020
</footer>
<link rel="preconnect" href="https://fonts.gstatic.com">
<link href="https://fonts.googleapis.com/css2?family=Source+Sans+Pro:ital,wght@0,200;0,300;0,400;0,600;0,700;0,900;1,200;1,300;1,400;1,600;1,700;1,900&display=swap" rel="stylesheet">
<link rel="stylesheet" href="/css/highlight/default.css">
<script src="/js/highlight.pack.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
</body>
</html>

111
src/blog/more-than-json.html.pm
View File

@ -1,111 +0,0 @@
#lang pollen
◊(define-meta title "More than JSON: ActivityPub and JSON-LD")
◊(define-meta published "2019-04-23")
◊blockquote{◊italic{In which our hero discovers the power of normalization and JSON-LD}}
◊heading{The problem with JSON}
I’ve been doing a lot of research for my current side project, ◊link[#:href "https://jeremydormitzer.com/blog/announcing-pterotype/"]{Pterotype}. It’s a new kind of social network built as a WordPress plugin that respects your freedom, encourages choice, and interoperates with existing social networks through the power of ◊link[#:href "https://jeremydormitzer.com/blog/what-is-activitypub-and-how-will-it-change-the-internet/"]{ActivityPub}. It’s undergone several iterations already – the beta has been out for a while now, and I’ve been working hard on a version 2 for the last several months.
One of the things I wasn’t satisfied with in the first version of Pterotype was the way it stores incoming data. ActivityPub messages are serialized in a dialect of JSON called ◊link[#:href "https://json-ld.org/"]{JSON-LD}. I didn’t really get JSON-LD when I started this project. It seems overcomplicated and confusing, and I was more interested in shipping something that worked than understanding the theoretical underpinnings of the federated web. So I just kept the incoming data in JSON format. This worked, sort of, but I kept running into annoying, hard-to-reason about situations. For example, consider this ActivityPub object, representing a new note that Sally published:
◊codeblock[#:lang "json"]{
{
"@context": "https://www.w3.org/ns/activitystreams",
"id": "https://example.org/activities/1",
"type": "Create",
"actor": {
"type": "Person",
"id": "https://example.org/sally",
"name": "Sally"
},
"object": {
"id": "https://example.org/notes/1",
"type": "Note",
"content": "This is a simple note"
},
"published": "2015-01-25T12:34:56Z"
}
}
The problem is that the above object, according to the ActivityPub specification, is semantically equivalent to this one:
◊codeblock[#:lang "json"]{
{
"@context": "https://www.w3.org/ns/activitystreams",
"id": "https://example.org/activities/1",
"type": "Create",
"actor": "https://example.org/sally",
"object": "https://example.org/notes/1",
"published": "2015-01-25T12:34:56Z"
}
}
This is the object graph in action – the ◊code{actor} and ◊code{object} properties are pointers to other objects, and as such they can either be JSON objects embedded within the ◊code{Create} activity, or URIs that dereference to the actual object (dereferencing is a fancy word for following the URI and replacing it with whatever JSON object is on the other side). Since I was representing these ActivityPub objects in this JSON format, that meant that whenever I saw an ◊code{actor} or ◊code{object} property, I always had to check whether it was an object or a URI and if it was a URI I had to dereference it to the proper object. This led to tons of annoying boilerplate and conditionals:
◊codeblock[#:lang "php"]{
if ( is_string( $activity['object'] ) ) {
$activity['object'] = dereference_object( $activity['object'] );
}
}
Yikes. So I came up with what I thought was a clever solution: just walk the object graph and dereference every URI I found whenever I saw a new JSON object. So I would receive Sally’s ◊code{Create} activity and traverse the JSON representation of its graph, dereferencing the ◊code{actor} and ◊code{object} objects in the process. This effectively turned the second representation above into the first one. Problem solved, right?
Well, not quite. There are actually a bunch of problems with that approach. First, not all URIs in the JSON object should be dereferenced. For example, there is an ActivityPub attribute called ◊code{url} that is – you guessed it – a URL! And it is supposed to stay a URL, not get dereferenced to some other thing. Okay, so I’ll only dereference URIs that belong to attributes I know should contain references to other objects – ◊code{actor}, ◊code{object}, etc. But there’s still a problem! There’s no guarantee that we’ll be able to successfully dereference a URI. Maybe the server that was hosting that object went down. Maybe there’s a temporary network failure. Maybe it’s the year 3000 and bitrot has taken down 80% of the internet. The point is, even if we preemptively dereference all the URIs we can, we still need to handle the case where we couldn’t access the actual object and are stuck with the URI. Which means we still need those stupid conditionals everywhere!
◊heading{JSON-LD to the rescue}
So what’s the actual solution for this? Well, as it turns out these were exactly the types of issues that JSON-LD is designed to solve. JSON-LD provides a way to normalize data into a standard form based on a ◊italic{context} that defines a schema for the data. Here’s the second version of Sally’s activity from above after undergoing JSON-LD expansion:
◊codeblock[#:lang "json"]{
[
{
"https://www.w3.org/ns/activitystreams#actor": [
{
"@id": "https://example.org/sally"
}
],
"@id": "https://example.org/activities/1",
"https://www.w3.org/ns/activitystreams#object": [
{
"@id": "https://example.org/notes/1"
}
],
"https://www.w3.org/ns/activitystreams#published": [
{
"@type": "http://www.w3.org/2001/XMLSchema#dateTime",
"@value": "2015-01-25T12:34:56Z"
}
],
"@type": [
"https://www.w3.org/ns/activitystreams#Create"
]
}
]
}
So what’s up with those weird URL-looking attributes? And why has everything become an array?
The expansion algorithm has normalized the data into a form that is supposed to be universally normalized. The attributes – ◊code{object}, ◊code{actor}, etc. – have become URIs with a universal meaning and a known schema. In other words, any application that speaks JSON-LD knows what an ◊code{https://www.w3.org/ns/activitystreams#actor} is, even if they don’t know what an actor is.
Importantly for our purposes, take a look at what the ◊code{object} field has turned into. We went from:
◊codeblock[#:lang "json"]{
"object": "https://example.org/notes/1"
}
To:
◊codeblock[#:lang "json"]{
"https://www.w3.org/ns/activitystreams#object": [
{
"@id": "https://example.org/notes/1"
}
]
}
Because the object attribute is specified in the ◊link[#:href "https://www.w3.org/ns/activitystreams.jsonld"]{ActivityStreams JSON-LD vocabulary} to be of ◊code{@type}: ◊code{@id}, the expansion process was able to infer that ◊code{object} ought to be, well, an object. This neatly solves the problem of “is this string attribute actually a reference” – all references are clearly marked by their ◊code{@id} attributes now. Plus, this allows us to be smarter about when we dereference an object – for example, we can defer dereferencing until we actually need to access the attributes of the linked object. This approach also addresses the problem of network errors when dereferencing – if we can’t dereference, we just end up with an object that has only an ◊code{@id}, which can still be handled gracefully by the application.
Hopefully this gave some insight into the types of challenges involved with building ActivityPub-powered applications and the point of JSON-LD. Have questions? Did I do something wrong? Let me know in the comments or on the ◊link[#:href "https://mastodon.technology/@jdormit"]{Fediverse}!

111
src/blog/more-than-json.md Normal file
View File

@ -0,0 +1,111 @@
---
title: "More than JSON: ActivityPub and JSON-LD"
author: Jeremy Dormitzer
published: 2019-04-23
---
> *In which our hero discovers the power of normalization and JSON-LD*
## The problem with JSON
I’ve been doing a lot of research for my current side project, [Pterotype](https://jeremydormitzer.com/blog/announcing-pterotype/). It’s a new kind of social network built as a WordPress plugin that respects your freedom, encourages choice, and interoperates with existing social networks through the power of [ActivityPub](https://jeremydormitzer.com/blog/what-is-activitypub-and-how-will-it-change-the-internet/). It’s undergone several iterations already – the beta has been out for a while now, and I’ve been working hard on a version 2 for the last several months.
One of the things I wasn’t satisfied with in the first version of Pterotype was the way it stores incoming data. ActivityPub messages are serialized in a dialect of JSON called [JSON-LD](https://json-ld.org/). I didn’t really get JSON-LD when I started this project. It seems overcomplicated and confusing, and I was more interested in shipping something that worked than understanding the theoretical underpinnings of the federated web. So I just kept the incoming data in JSON format. This worked, sort of, but I kept running into annoying, hard-to-reason about situations. For example, consider this ActivityPub object, representing a new note that Sally published:
```json
{
"@context": "https://www.w3.org/ns/activitystreams",
"id": "https://example.org/activities/1",
"type": "Create",
"actor": {
"type": "Person",
"id": "https://example.org/sally",
"name": "Sally"
},
"object": {
"id": "https://example.org/notes/1",
"type": "Note",
"content": "This is a simple note"
},
"published": "2015-01-25T12:34:56Z"
}
```
The problem is that the above object, according to the ActivityPub specification, is semantically equivalent to this one:
```json
{
"@context": "https://www.w3.org/ns/activitystreams",
"id": "https://example.org/activities/1",
"type": "Create",
"actor": "https://example.org/sally",
"object": "https://example.org/notes/1",
"published": "2015-01-25T12:34:56Z"
}
```
This is the object graph in action – the `actor` and `object` properties are pointers to other objects, and as such they can either be JSON objects embedded within the `Create` activity, or URIs that dereference to the actual object (dereferencing is a fancy word for following the URI and replacing it with whatever JSON object is on the other side). Since I was representing these ActivityPub objects in this JSON format, that meant that whenever I saw an `actor` or `object` property, I always had to check whether it was an object or a URI and if it was a URI I had to dereference it to the proper object. This led to tons of annoying boilerplate and conditionals:
```php
if ( is_string( $activity['object'] ) ) {
$activity['object'] = dereference_object( $activity['object'] );
}
```
Yikes. So I came up with what I thought was a clever solution: just walk the object graph and dereference every URI I found whenever I saw a new JSON object. So I would receive Sally’s `Create` activity and traverse the JSON representation of its graph, dereferencing the `actor` and `object` objects in the process. This effectively turned the second representation above into the first one. Problem solved, right?
Well, not quite. There are actually a bunch of problems with that approach. First, not all URIs in the JSON object should be dereferenced. For example, there is an ActivityPub attribute called `url` that is – you guessed it – a URL! And it is supposed to stay a URL, not get dereferenced to some other thing. Okay, so I’ll only dereference URIs that belong to attributes I know should contain references to other objects – `actor`, `object`, etc. But there’s still a problem! There’s no guarantee that we’ll be able to successfully dereference a URI. Maybe the server that was hosting that object went down. Maybe there’s a temporary network failure. Maybe it’s the year 3000 and bitrot has taken down 80% of the internet. The point is, even if we preemptively dereference all the URIs we can, we still need to handle the case where we couldn’t access the actual object and are stuck with the URI. Which means we still need those stupid conditionals everywhere!
## JSON-LD to the rescue
So what’s the actual solution for this? Well, as it turns out these were exactly the types of issues that JSON-LD is designed to solve. JSON-LD provides a way to normalize data into a standard form based on a /context/ that defines a schema for the data. Here’s the second version of Sally’s activity from above after undergoing JSON-LD expansion:
```json
[
{
"https://www.w3.org/ns/activitystreams#actor": [
{
"@id": "https://example.org/sally"
}
],
"@id": "https://example.org/activities/1",
"https://www.w3.org/ns/activitystreams#object": [
{
"@id": "https://example.org/notes/1"
}
],
"https://www.w3.org/ns/activitystreams#published": [
{
"@type": "http://www.w3.org/2001/XMLSchema#dateTime",
"@value": "2015-01-25T12:34:56Z"
}
],
"@type": [
"https://www.w3.org/ns/activitystreams#Create"
]
}
]
```
So what’s up with those weird URL-looking attributes? And why has everything become an array?
The expansion algorithm has normalized the data into a form that is supposed to be universally normalized. The attributes – `object`, `actor`, etc. – have become URIs with a universal meaning and a known schema. In other words, any application that speaks JSON-LD knows what an `https://www.w3.org/ns/activitystreams#actor` is, even if they don’t know what an actor is.
Importantly for our purposes, take a look at what the `object` field has turned into. We went from:
```json
"object": "https://example.org/notes/1"
```
To:
```json
"https://www.w3.org/ns/activitystreams#object": [
{
"@id": "https://example.org/notes/1"
}
]
```
Because the object attribute is specified in the [ActivityStreams JSON-LD vocabulary](https://www.w3.org/ns/activitystreams.jsonld) to be of `@type`: `@id`, the expansion process was able to infer that `object` ought to be, well, an object. This neatly solves the problem of “is this string attribute actually a reference” – all references are clearly marked by their `@id` attributes now. Plus, this allows us to be smarter about when we dereference an object – for example, we can defer dereferencing until we actually need to access the attributes of the linked object. This approach also addresses the problem of network errors when dereferencing – if we can’t dereference, we just end up with an object that has only an `@id`, which can still be handled gracefully by the application.
Hopefully this gave some insight into the types of challenges involved with building ActivityPub-powered applications and the point of JSON-LD. Have questions? Did I do something wrong? Let me know in the comments or on the [Fediverse](https://mastodon.technology/@jdormit)!

25
src/blog/unifydb-dev-diary-0-intro.html.pm
View File

@ -1,25 +0,0 @@
#lang pollen
◊(define-meta title "unifyDB Dev Diary 0: I’m building a database!")
◊(define-meta published "2020-08-09")
Phew, it’s been a while! Over a year, in fact. And what a wild year! Lots of good things happened: I got married, got a new job that I love, moved to a nice new apartment. Also some not-so-nice things, but since you are all living through 2020 just like me I don’t think I need to go into those. But I have still found some side-project time, and I’d like to start talking about what I’m building.
So – I’m excited to announce that I’m building a database! I’m calling it unifyDB. It’s going to be a general-purpose database with some pretty interesting properties:
◊ul{
◊li{It maintains a complete history of changes to all entities stored in the database}
◊li{You can make queries for historical data, e.g. “what did this user record look like last Tuesday?”}
◊li{Arbitrary metadata can be attached to transactions – for example, you can add an application user ID to every transaction your app makes}
◊li{Fine-grained access control is built into the database, allowing developers to limit access to particular attributes across all entities}
}
This is the database that I’ve always wanted – basically, I’m tired of being in meetings where the boss says “who changed this user’s email address?” and everyone just looks at each other and shrugs.
I’m designing unifyDB to be as modular as possible – I want it to be as easy to run it as a single node on your local machine as it is to run in an autoscaling cluster on your cloud of choice.
I’ve actually been working on this on and off for over a year. The code lives in ◊link[#:href "https://github.com/unifydb/unifydb" #:target "_blank"]{a GitHub repository}. Fair warning: it’s mostly undocumented and nowhere close to being finished. So far, I’ve written the query engine, the transaction handler, the web server (yes, it has an HTTP interface), and a bunch of underlying infrastructure. So as it currently stands, unifyDB is able to store data (in-memory since I haven’t built the storage layer yet) and issue history-aware queries. I’m in the middle of writing the authentication mechanism. After that, it’s on to the storage layer, then most likely the access control layer.
I’m going to start publishing monthly development diaries detailing the more interesting aspects of database. I’ll start with a post about the query system implementation sometime in the next couple of weeks. Sound interesting? Follow along ◊link[#:href "https://feedly.com/i/subscription/feed%2Fhttps%3A%2F%2Fjeremydormitzer.com%2Fblog%2Ffeed.xml" #:target "_blank"]{on Feedly} or ◊link[#:href "https://jeremydormitzer.com/blog/feed.xml" #:target "_blank"]{your RSS reader of choice}!.
In the meantime, if you want to know more about unifyDB or just want to geek out, hit me up ◊link[#:href "https://twitter.com/jeremydormitzer" #:target "_blank"]{on Twitter}.

23
src/blog/unifydb-dev-diary-0-intro.md Normal file
View File

@ -0,0 +1,23 @@
---
title: "unifyDB Dev Diary 0: I’m building a database!"
author: Jeremy Dormitzer
published: 2020-08-09
---
Phew, it’s been a while! Over a year, in fact. And what a wild year! Lots of good things happened: I got married, got a new job that I love, moved to a nice new apartment. Also some not-so-nice things, but since you are all living through 2020 just like me I don’t think I need to go into those. But I have still found some side-project time, and I’d like to start talking about what I’m building.
So – I’m excited to announce that I’m building a database! I’m calling it unifyDB. It’s going to be a general-purpose database with some pretty interesting properties:
- It maintains a complete history of changes to all entities stored in the database
- You can make queries for historical data, e.g. “what did this user record look like last Tuesday?”
- Arbitrary metadata can be attached to transactions – for example, you can add an application user ID to every transaction your app makes
- Fine-grained access control is built into the database, allowing developers to limit access to particular attributes across all entities
This is the database that I’ve always wanted – basically, I’m tired of being in meetings where the boss says “who changed this user’s email address?” and everyone just looks at each other and shrugs.
I’m designing unifyDB to be as modular as possible – I want it to be as easy to run it as a single node on your local machine as it is to run in an autoscaling cluster on your cloud of choice.
I’ve actually been working on this on and off for over a year. The code lives in [a GitHub repository](https://github.com/unifydb/unifydb). Fair warning: it’s mostly undocumented and nowhere close to being finished. So far, I’ve written the query engine, the transaction handler, the web server (yes, it has an HTTP interface), and a bunch of underlying infrastructure. So as it currently stands, unifyDB is able to store data (in-memory since I haven’t built the storage layer yet) and issue history-aware queries. I’m in the middle of writing the authentication mechanism. After that, it’s on to the storage layer, then most likely the access control layer.
I’m going to start publishing monthly development diaries detailing the more interesting aspects of database. I’ll start with a post about the query system implementation sometime in the next couple of weeks. Sound interesting? Follow along [on Feedly](https://feedly.com/i/subscription/feed%2Fhttps%3A%2F%2Fjeremydormitzer.com%2Fblog%2Ffeed.xml) or [your RSS reader of choice](https://jeremydormitzer.com/blog/feed.xml)!.
In the meantime, if you want to know more about unifyDB or just want to geek out, hit me up [on Twitter](https://twitter.com/jeremydormitzer).

117
src/blog/unifydb-dev-diary-1-query.html.pm
View File

@ -1,117 +0,0 @@
#lang pollen
◊(define-meta title "unifyDB Dev Diary 1: the query system")
◊(define-meta published "2020-10-03")
This is the first development diary for the database I'm writing, ◊link[#:href "https://github.com/unifydb/unifydb" #:target "_blank"]{unifyDB}. I wrote a brief introduction to the project ◊link[#:href "https://jeremydormitzer.com/blog/unifydb-dev-diary-0-intro.html" #:target "_blank"]{here}. In this post I'm going to talk about unifyDB's query system: what it does and how it works.
I want to start with an example of a unifyDB query, but to understand that we need to understand a bit about how unifyDB represents data. All data in unifyDB is stored as a collection of facts. A fact is a tuple with three pieces of information: an entity ID, an attribute name, and a value (actually, a fact has two additional fields, a transaction ID and an ◊code{added?} flag, but we won't worry about those until we talk about time-traveling queries, which deserves its own blog post). For example, we might represent some user records with the following set of facts:
◊codeblock{
(1, "username", "harry")
(1, "role", "user")
(1, "preferred-theme", "light")
(2, "username", "dumbledore")
(2, "role", "user")
(2, "role", "admin")
(2, "preferred-theme", "light")
(3, "username", "you-know-who")
(3, "role", "user")
(3, "role", "user")
(3, "role", "admin")
(3, "preferred-theme", "dark")
}
This corresponds with the following records in a more conventional JSON format:
◊codeblock[#:lang "json"]{
[
{
"id": 1,
"username": "harry",
"role": ["user"],
"preferred-theme": "light"
},
{
"id": 2,
"username": "dumbledore",
"role": ["user", "admin"],
"preferred-theme": "light"
},
{
"id": 3,
"username": "you-know-who",
"role": ["user", "admin"],
"preferred-theme": "dark"
}
]
}
(The astute reader will notice that there’s not actually a way to specify using a set of facts that ◊code{"role"} is a list but ◊code{"preferred-theme"} is a scalar value, i.e. the cardinality of an attribute. This requires another database feature, attribute schemas, that I’m going to save for another blog post.)
With that under our belt, let's take a look at an example unifyDB query. The unifyDB server understands query written in ◊link[#:href "https://github.com/edn-format/edn" #:target "_blank"]{extensible data notation}, but database clients for different programming languages will allow developers to write queries that feel native to that language. Here's a query in EDN format:
◊codeblock[#:lang "clojure"]{
{:find [?username]
:where [[?e :preferred-theme "light"]
[?e :username ?username]]}
}
This query says, “find me the values of all the ◊code{username} attributes of entities whose ◊code{preferred-theme} is ◊code{"light"}”. If we run this query on the set of facts given above, it would return:
◊codeblock[#:lang "clojure"]{
[["harry"]
["dumbledore"]]
}
Note that the return value is a list of lists — although our query only asked for one field, ◊code{username}, it could have asked for more, in which case each result in the result list would be a list with all the requested values. Once again, although unifyDB itself returns data in EDN format, client libraries will wrap that return value in whatever native data structure is convenient.
Let’s break that query down a bit. First, a bit of notation: any symbol that starts with a ◊code{?} is called a variable, and is similar in spirit to a variable in a programming language. The query above has two major pieces: a ◊code{:find} clause and a ◊code{:where} clause. The ◊code{:find} clause is straightforward: it asks to find the value of the variable ◊code{?username}. But how does it know what value that variable has? That’s where things get interesting.
Let's take a closer look at the ◊code{:where} clause:
◊codeblock[#:lang "clojure"]{
:where [[?e :preferred-theme "light"]
[?e :username ?username]]
}
It is a list of two relations - that is, expressions which assert some relationship between variables. The first relation, ◊code{[?e :preferred-theme "light"]}, declares that there is some entity ◊code{?e} whose ◊code{:preferred-theme} attribute has value ◊code{"light"}. The second relation is slightly more abstract, declaring a relation between some entity ◊code{?e} and the value of its ◊code{:username} attribute, which it assigns to the variable ◊code{?username}.
Notice that both relations share a variable, ◊code{?e}. This is where the magic happens! When two relations share a variable, they are said to ◊em{unify}. This means that the query engine finds all facts that satisfy ◊em{both} relations for some entity ◊code{?e}. In other words, unifyDB will find all sets of facts such that the facts share an entity ◊code{?e}, have one fact with attribute ◊code{:preferred-theme} and value ◊code{"light"}, and have another fact with attribute ◊code{:username} and any value.
The result of this unification process is a set of variable bindings, calculated from the facts that satisfy the query relation. In our example, we find that the following set of facts satisfies the query relation:
◊codeblock{
(1, "username", "harry")
(1, "preferred-theme", "light")
(2, "username", "dumbledore")
(2, "preferred-theme", "light")
}
Unifying these facts with the variables in the ◊code{:where} clause yields the following set of bindings:
◊codeblock{
{
?e: 1,
?username: "harry"
},
{
?e: 2,
?username: "dumbledore"
}
}
Finally, since our ◊code{:find} clause asks only for the variable ◊code{?username}, we look up that variable in the binding set, returning one result for each binding in the set:
◊codeblock[#:lang "clojure"]{
[["harry"]
["dumbledore"]]
}
This unification approach to querying makes the database particularly powerful. Although in this example we unified on the entity ID, we can also unify on the attribute name, value, or some combination of all three. This gives unifyDB the ability to function as a document store (looking up the “documents”, i.e. entities, which have attributes and values matching some pattern); or as a column-oriented database, looking for all the values of a certain attribute or even all the attributes that have a certain value. Of course, most apps will use a combination of all these different querying approaches, letting the database work for them in whatever way they need for a particular feature.
In fact, this is only half of the query engine, since it also supports adding ◊em{rules} that let you compute new facts from existing facts in the database, but that is complex enough to warrant its own post.
There is a lot more I could write about here, but this is running kind of long so I’m going to leave it at this for now. You can follow the development of unifyDB ◊link[#:href "https://github.com/unifydb/unifydb/" #:target "_blank"]{on GitHub} (the query engine is implemented ◊link[#:href "https://github.com/unifydb/unifydb/blob/master/src/unifydb/query.clj" #:target "_blank"]{here} and unification is implemented ◊link[#:href "https://github.com/unifydb/unifydb/blob/master/src/unifydb/unify.clj" #:target "_blank"]{here}). If you are interested in this topic and want to dive into the implementation, I based my work on the excellent logical database engine in ◊link[#:href "https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-29.html#%_sec_4.4" #:target "_blank"]{chapter 4.4 of Structure and Interpretation of Computer Programs}.
As always, if you want to know more about unifyDB, have questions about this post or just want to geek out, hit me up ◊link[#:href "https://twitter.com/jeremydormitzer" #:target "_blank"]{on Twitter}.

117
src/blog/unifydb-dev-diary-1-query.md Normal file
View File

@ -0,0 +1,117 @@
---
title: "unifyDB Dev Diary 1: the query system"
author: Jeremy Dormitzer
published: 2020-10-03
---
This is the first development diary for the database I'm writing, [unifyDB](https://github.com/unifydb/unifydb). I wrote a brief introduction to the project [here](https://jeremydormitzer.com/blog/unifydb-dev-diary-0-intro.html). In this post I'm going to talk about unifyDB's query system: what it does and how it works.
I want to start with an example of a unifyDB query, but to understand that we need to understand a bit about how unifyDB represents data. All data in unifyDB is stored as a collection of facts. A fact is a tuple with three pieces of information: an entity ID, an attribute name, and a value (actually, a fact has two additional fields, a transaction ID and an `added?` flag, but we won't worry about those until we talk about time-traveling queries, which deserves its own blog post). For example, we might represent some user records with the following set of facts:
```
(1, "username", "harry")
(1, "role", "user")
(1, "preferred-theme", "light")
(2, "username", "dumbledore")
(2, "role", "user")
(2, "role", "admin")
(2, "preferred-theme", "light")
(3, "username", "you-know-who")
(3, "role", "user")
(3, "role", "user")
(3, "role", "admin")
(3, "preferred-theme", "dark")
```
This corresponds with the following records in a more conventional JSON format:
```json
[
{
"id": 1,
"username": "harry",
"role": ["user"],
"preferred-theme": "light"
},
{
"id": 2,
"username": "dumbledore",
"role": ["user", "admin"],
"preferred-theme": "light"
},
{
"id": 3,
"username": "you-know-who",
"role": ["user", "admin"],
"preferred-theme": "dark"
}
]
```
(The astute reader will notice that there’s not actually a way to specify using a set of facts that `"role"` is a list but `"preferred-theme"` is a scalar value, i.e. the cardinality of an attribute. This requires another database feature, attribute schemas, that I’m going to save for another blog post.)
With that under our belt, let's take a look at an example unifyDB query. The unifyDB server understands query written in [extensible data notation](https://github.com/edn-format/edn), but database clients for different programming languages will allow developers to write queries that feel native to that language. Here's a query in EDN format:
```clojure
{:find [?username]
:where [[?e :preferred-theme "light"]
[?e :username ?username]]}
```
This query says, “find me the values of all the `username` attributes of entities whose `preferred-theme` is `"light"`”. If we run this query on the set of facts given above, it would return:
```clojure
[["harry"]
["dumbledore"]]
```
Note that the return value is a list of lists — although our query only asked for one field, `username`, it could have asked for more, in which case each result in the result list would be a list with all the requested values. Once again, although unifyDB itself returns data in EDN format, client libraries will wrap that return value in whatever native data structure is convenient.
Let’s break that query down a bit. First, a bit of notation: any symbol that starts with a `?` is called a variable, and is similar in spirit to a variable in a programming language. The query above has two major pieces: a `:find` clause and a `:where` clause. The `:find` clause is straightforward: it asks to find the value of the variable `?username`. But how does it know what value that variable has? That’s where things get interesting.
Let's take a closer look at the `:where` clause:
```clojure
:where [[?e :preferred-theme "light"]
[?e :username ?username]]
```
It is a list of two relations - that is, expressions which assert some relationship between variables. The first relation, `[?e :preferred-theme "light"]`, declares that there is some entity `?e` whose `:preferred-theme` attribute has value `"light"`. The second relation is slightly more abstract, declaring a relation between some entity `?e` and the value of its `:username` attribute, which it assigns to the variable `?username`.
Notice that both relations share a variable, `?e`. This is where the magic happens! When two relations share a variable, they are said to *unify*. This means that the query engine finds all facts that satisfy *both* relations for some entity `?e`. In other words, unifyDB will find all sets of facts such that the facts share an entity `?e`, have one fact with attribute `:preferred-theme` and value `"light"`, and have another fact with attribute `:username` and any value.
The result of this unification process is a set of variable bindings, calculated from the facts that satisfy the query relation. In our example, we find that the following set of facts satisfies the query relation:
```
(1, "username", "harry")
(1, "preferred-theme", "light")
(2, "username", "dumbledore")
(2, "preferred-theme", "light")
```
Unifying these facts with the variables in the `:where` clause yields the following set of bindings:
```
{
?e: 1,
?username: "harry"
},
{
?e: 2,
?username: "dumbledore"
}
```
Finally, since our `:find` clause asks only for the variable `?username`, we look up that variable in the binding set, returning one result for each binding in the set:
```clojure
[["harry"]
["dumbledore"]]
```
This unification approach to querying makes the database particularly powerful. Although in this example we unified on the entity ID, we can also unify on the attribute name, value, or some combination of all three. This gives unifyDB the ability to function as a document store (looking up the “documents”, i.e. entities, which have attributes and values matching some pattern); or as a column-oriented database, looking for all the values of a certain attribute or even all the attributes that have a certain value. Of course, most apps will use a combination of all these different querying approaches, letting the database work for them in whatever way they need for a particular feature.
In fact, this is only half of the query engine, since it also supports adding *rules* that let you compute new facts from existing facts in the database, but that is complex enough to warrant its own post.
There is a lot more I could write about here, but this is running kind of long so I’m going to leave it at this for now. You can follow the development of unifyDB [on GitHub](https://github.com/unifydb/unifydb/) (the query engine is implemented [here](https://github.com/unifydb/unifydb/blob/master/src/unifydb/query.clj) and unification is implemented [here](https://github.com/unifydb/unifydb/blob/master/src/unifydb/unify.clj)). If you are interested in this topic and want to dive into the implementation, I based my work on the excellent logical database engine in [chapter 4.4 of Structure and Interpretation of Computer Programs](https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-29.html#%_sec_4.4).
As always, if you want to know more about unifyDB, have questions about this post or just want to geek out, hit me up [on Twitter](https://twitter.com/jeremydormitzer).

38
src/blog/what-is-activitypub.html.pm → src/blog/what-is-activitypub.md
View File

@ -1,45 +1,45 @@
#lang pollen
---
title: What is ActivityPub, and how will it change the internet?
author: Jeremy Dormitzer
published: 2018-09-15
---
<img alt="a network-shaped board game" src="/images/board-game.jpg" />
◊(define-meta title "What is ActivityPub, and how will it change the internet?")
◊(define-meta published "2018-09-15")
## A new kind of social network
◊header-image[#:src "/images/board-game.jpg"]
There’s a new social network in town. It’s called [Mastodon](https://joinmastodon.org/). You might have even heard of it. On the surface, Mastodon feels a lot like Twitter: you post “toots” up to 500 characters; you follow other users who say interesting things; you can favorite a toot or re-post it to your own followers. But Mastodon is different from Twitter in some fundamental ways. It offers many more ways for users to control the posts they see. It fosters awareness of the effect your posts have on others through a content warning system and encourages accessibility with captioned images. At its core, though, there’s a more fundamental difference from existing social networks: Mastodon isn’t controlled by a single corporation. Anyone can operate a Mastodon server, and users on any server can interact with users on any other Mastodon server.
◊heading{A new kind of social network}
There’s a new social network in town. It’s called ◊link[#:href "https://joinmastodon.org/"]{Mastodon}. You might have even heard of it. On the surface, Mastodon feels a lot like Twitter: you post “toots” up to 500 characters; you follow other users who say interesting things; you can favorite a toot or re-post it to your own followers. But Mastodon is different from Twitter in some fundamental ways. It offers many more ways for users to control the posts they see. It fosters awareness of the effect your posts have on others through a content warning system and encourages accessibility with captioned images. At its core, though, there’s a more fundamental difference from existing social networks: Mastodon isn’t controlled by a single corporation. Anyone can operate a Mastodon server, and users on any server can interact with users on any other Mastodon server.
This decentralized model is called federation. Email is a good analogy here: I can have a Gmail account and you can have an Outlook account, but we can still send mail to each other. In the same way, I can have an account on ◊link[#:href "https://mastodon.technology/invite/JguyVqcL"]{mastodon.technology}, and you can have an account on ◊link[#:href "https://mastodon.social/"]{mastodon.social}, but we can still follow each other, like and re-post each other’s toots, and @mention each other. Just like Gmail servers know how to talk to Outlook servers, Mastodon servers know how to talk to other Mastodon servers (if you hear people talking about a Mastodon “instance”, they mean server). And just like Gmail and Outlook are controlled by different corporations, Mastodon servers are owned and operated by many different people and organizations. If you wanted, you could ◊link[#:href "https://github.com/tootsuite/documentation#running-mastodon"]{host your own Mastodon instance}!
This decentralized model is called federation. Email is a good analogy here: I can have a Gmail account and you can have an Outlook account, but we can still send mail to each other. In the same way, I can have an account on [mastodon.technology](https://mastodon.technology/invite/JguyVqcL), and you can have an account on [mastodon.social](https://mastodon.social/), but we can still follow each other, like and re-post each other’s toots, and @mention each other. Just like Gmail servers know how to talk to Outlook servers, Mastodon servers know how to talk to other Mastodon servers (if you hear people talking about a Mastodon “instance”, they mean server). And just like Gmail and Outlook are controlled by different corporations, Mastodon servers are owned and operated by many different people and organizations. If you wanted, you could [host your own Mastodon instance](https://github.com/tootsuite/documentation#running-mastodon)!
Why does this matter? It means that Mastodon users have choice about where they hang out online. If Twitter decides that your posts shouldn’t be on their platform, they can shut down your account and there’s nothing you can do about it (or conversely, if they decide your f-ed up content is totally fine, there’s nothing anyone else can do about it). On the other hand, if you disagree with the administrators of your Mastodon instance, you have the choice to move your account to another instance (switching providers, as it were) or to host your own instance if you’re willing to dedicate the time and effort.
The federated model also tends to align incentives better than centralized alternatives. Mastodon instances are usually run and moderated by members of the community that uses that particular Mastodon server – for example, I’m part of a community of tech folks over at ◊link[#:href "https://mastodon.technology/invite/JguyVqcL"]{mastodon.technology}, and our server is administrated and moderated by a member of the community. He has a vested interest in making mastodon.technology a nice place to hang out since he hangs out there too. Contrast that with Twitter: Twitter is owned and operated by a massive, venture-backed, for-profit corporation. Now, I’m certainly not against companies making money (more on that later), but Twitter only cares about making Twitter a nice place to hang out to the extent that they profit by it, which has led them to make ◊link[#:href "http://www.slate.com/articles/technology/cover_story/2017/03/twitter_s_timeline_algorithm_and_its_effect_on_us_explained.html"]{some user-unfriendly choices}.
The federated model also tends to align incentives better than centralized alternatives. Mastodon instances are usually run and moderated by members of the community that uses that particular Mastodon server – for example, I’m part of a community of tech folks over at [mastodon.technology](https://mastodon.technology/invite/JguyVqcL), and our server is administrated and moderated by a member of the community. He has a vested interest in making mastodon.technology a nice place to hang out since he hangs out there too. Contrast that with Twitter: Twitter is owned and operated by a massive, venture-backed, for-profit corporation. Now, I’m certainly not against companies making money (more on that later), but Twitter only cares about making Twitter a nice place to hang out to the extent that they profit by it, which has led them to make [some user-unfriendly choices](http://www.slate.com/articles/technology/cover_story/2017/03/twitter_s_timeline_algorithm_and_its_effect_on_us_explained.html).
So Mastodon is pretty cool. But that’s not what really gets me excited. I’m excited about how Mastodon servers allow users on different instances to interact. It’s a protocol called ◊link[#:href "https://activitypub.rocks/"]{ActivityPub}, and it’s going to change the way the internet works.
So Mastodon is pretty cool. But that’s not what really gets me excited. I’m excited about how Mastodon servers allow users on different instances to interact. It’s a protocol called [ActivityPub](https://activitypub.rocks/), and it’s going to change the way the internet works.
◊heading{ActivityPub}
## ActivityPub
ActivityPub is a social networking protocol. Think of it as a language that describes social networks: the nouns are users and posts, and the verbs are like, follow, share, create… ActivityPub gives applications a shared vocabulary that they can use to communicate with each other. If a server implements ActivityPub, it can publish posts that any other server that implements ActivityPub knows how to share, like and reply to. It can also share, like, or reply to posts from other servers that speak ActivityPub on behalf of its users.
This is how Mastodon instances let users interact with users on other instances: because every Mastodon instance implements ActivityPub, one instance knows how to interpret a post published from another instance, how to like a post from another instance, how to follow a user from another instance, etc.
ActivityPub is much bigger than just Mastodon, though. It’s a language that any application can implement. For example, there’s a YouTube clone called ◊link[#:href "https://joinpeertube.org/en/faq/"]{PeerTube} that also implements ActivityPub. Because it speaks the same language as Mastodon, a Mastodon user can follow a PeerTube user. If the PeerTube user posts a new video, it will show up in the Mastodon user’s feed. The Mastodon user can comment on the PeerTube video directly from Mastodon. Think about that for a second. Any app that implements ActivityPub becomes part of a massive social network, one that conserves user choice and tears down walled gardens. Imagine if you could log into Facebook and see posts from your friends on Instagram and Twitter, without needing an Instagram or Twitter account.
ActivityPub is much bigger than just Mastodon, though. It’s a language that any application can implement. For example, there’s a YouTube clone called [PeerTube](https://joinpeertube.org/en/faq/) that also implements ActivityPub. Because it speaks the same language as Mastodon, a Mastodon user can follow a PeerTube user. If the PeerTube user posts a new video, it will show up in the Mastodon user’s feed. The Mastodon user can comment on the PeerTube video directly from Mastodon. Think about that for a second. Any app that implements ActivityPub becomes part of a massive social network, one that conserves user choice and tears down walled gardens. Imagine if you could log into Facebook and see posts from your friends on Instagram and Twitter, without needing an Instagram or Twitter account.
So here’s how ActivityPub is going to change the internet:
◊heading{No more walled gardens}
## No more walled gardens
ActivityPub separates content from platform. Posts from one platform propagate to other platforms, and users don’t need to make separate accounts on every platform that they want to use. This has an additional benefit: since your ActivityPub identity (your Mastodon account, your PeerTube account, etc.) is valid across all ActivityPub-compliant applications, it serves as a much stronger identity signal, preventing malicious actors from impersonating you (e.g. creating a Twitter account in your name). If you can share one account across multiple platforms, no one can pretend to be you on those platforms – you are already there!
◊heading{Social networking comes built-in}
## Social networking comes built-in
With traditional internet media, you need to rely on external services to share your work on social networks. If you want people to share your YouTube video around, you need to post it to Facebook or Twitter. But ActvityPub-enabled applications are social by nature. A PeerTube video can be shared or liked by default by users on Mastodon. A Plume blogger can build an audience on Mastodon or PeerTube without any additional effort since Mastodon and PeerTube users can follow Plume blogs natively. Users on all these platforms see content from the other apps on the platform of their choice. And Mastodon, PeerTube, and Plume are just the beginning – as more platforms begin implementing ActivityPub, the federated network grows exponentially.
◊heading{Network effects that help users instead of harming them}
## Network effects that help users instead of harming them
“Network effects” leaves kind of a dirty taste in my mouth. It’s usually used as a euphemism for “vendor lock-in”; the reason that Facebook became such a giant was that everyone needed to be on Facebook to participate in Facebook’s network. However, ActivityPub flips this equation on its head. As more platforms become ActivityPub compliant, it becomes more valuable for platforms implement ActivityPub: more apps means more users on the federated network, more posts to read and share, and more choice for users. This network effect discourages vendor lock-in. In the end, the users win.
◊heading{It’s going to be an uphill battle}
## It’s going to be an uphill battle
I hope I’ve convinced you of the radical impact that ActivityPub could have on the internet. But there are some significant barriers preventing widespread adoption. The thorniest one is money.
@ -47,12 +47,10 @@ Why is money an issue? Aren’t Mastodon and PeerTube free and open-source? Well
I know this isn’t necessarily a popular view in the open-source world, but I see funding as a critical priority to bring about the vision that ActivityPub promises. Unfortunately, it’s not clear how to obtain it.
All the major ActivityPub-compliant applications I’ve written about are open source projects, built and run by volunteers with ◊link[#:href "https://www.patreon.com/mastodon"]{tiny budgets}. Traditional social networking companies like Twitter and Facebook are funded by selling advertisements on their platform. But in order to make any significant revenue from ads, you need a centralized audience whose attention you control. Facebook needs to be able to say, “we have X billion users; give us your money and we will show them your ads”. Plus, the big social companies extract value from their users by segmenting them based on their behavior and interests, enabling micro-targeted ad campaigns.
All the major ActivityPub-compliant applications I’ve written about are open source projects, built and run by volunteers with [tiny budgets](https://www.patreon.com/mastodon). Traditional social networking companies like Twitter and Facebook are funded by selling advertisements on their platform. But in order to make any significant revenue from ads, you need a centralized audience whose attention you control. Facebook needs to be able to say, “we have X billion users; give us your money and we will show them your ads”. Plus, the big social companies extract value from their users by segmenting them based on their behavior and interests, enabling micro-targeted ad campaigns.
None of that is possible when the users and content are spread across many servers and platforms. There is no centralized audience to segment and advertise to. We’ll need to rethink the fundamental business model of social networking if we want ActivityPub to take off.
That being said, I do think ActivityPub offers tremendous business value. It turns your corporate blog into a social network by offering native sharing, following, liking, and replying. And it does so on your customer’s terms, which not only prevents abusive, spammy content but also helps your company’s reputation with users and potential customers. These benefits are valuable, and I think there is a way to turn that value into funding.
It’s important to think about how to make this revolution happen. ActivityPub has the potential to change the way we think and act on the internet, in a way that encourages decentralization and puts users first again. That’s a vision worth fighting for.
◊(define-meta tags '("activitypub"))

0
src/js/highlight/styles/a11y-dark.css.p → src/css/highlight/a11y-dark.css
View File

0
src/js/highlight/styles/a11y-light.css.p → src/css/highlight/a11y-light.css
View File

4
src/js/highlight/styles/agate.css.p → src/css/highlight/agate.css
View File

@ -1,6 +1,6 @@
/*!
* Agate by Taufik Nurrohman <https://github.com/tovic>
* ----------------------------------------------------
* Agate by Taufik Nurrohman <https://github.com/taufik-nurrohman>
* ---------------------------------------------------------------
*
* #ade5fc
* #a2fca2

0
src/js/highlight/styles/an-old-hope.css.p → src/css/highlight/an-old-hope.css
View File

0
src/js/highlight/styles/androidstudio.css.p → src/css/highlight/androidstudio.css
View File

1
src/js/highlight/styles/arduino-light.css.p → src/css/highlight/arduino-light.css
View File

@ -67,7 +67,6 @@ Arduino® Light Theme - Stefania Mellai <s.mellai@arduino.cc>
}
.hljs-meta {
color: #728E00;
color: #434f54;
}

0
src/js/highlight/styles/arta.css.p → src/css/highlight/arta.css
View File

0
src/js/highlight/styles/ascetic.css.p → src/css/highlight/ascetic.css
View File

0
src/js/highlight/styles/atelier-cave-dark.css.p → src/css/highlight/atelier-cave-dark.css
View File

0
src/js/highlight/styles/atelier-cave-light.css.p → src/css/highlight/atelier-cave-light.css
View File

0
src/js/highlight/styles/atelier-dune-dark.css.p → src/css/highlight/atelier-dune-dark.css
View File

0
src/js/highlight/styles/atelier-dune-light.css.p → src/css/highlight/atelier-dune-light.css
View File

0
src/js/highlight/styles/atelier-estuary-dark.css.p → src/css/highlight/atelier-estuary-dark.css
View File

0
src/js/highlight/styles/atelier-estuary-light.css.p → src/css/highlight/atelier-estuary-light.css
View File

0
src/js/highlight/styles/atelier-forest-dark.css.p → src/css/highlight/atelier-forest-dark.css
View File

0
src/js/highlight/styles/atelier-forest-light.css.p → src/css/highlight/atelier-forest-light.css
View File

0
src/js/highlight/styles/atelier-heath-dark.css.p → src/css/highlight/atelier-heath-dark.css
View File

0
src/js/highlight/styles/atelier-heath-light.css.p → src/css/highlight/atelier-heath-light.css
View File

0
src/js/highlight/styles/atelier-lakeside-dark.css.p → src/css/highlight/atelier-lakeside-dark.css
View File

0
src/js/highlight/styles/atelier-lakeside-light.css.p → src/css/highlight/atelier-lakeside-light.css
View File

0
src/js/highlight/styles/atelier-plateau-dark.css.p → src/css/highlight/atelier-plateau-dark.css
View File

0
src/js/highlight/styles/atelier-plateau-light.css.p → src/css/highlight/atelier-plateau-light.css
View File

0
src/js/highlight/styles/atelier-savanna-dark.css.p → src/css/highlight/atelier-savanna-dark.css
View File

0
src/js/highlight/styles/atelier-savanna-light.css.p → src/css/highlight/atelier-savanna-light.css
View File

0
src/js/highlight/styles/atelier-seaside-dark.css.p → src/css/highlight/atelier-seaside-dark.css
View File

0
src/js/highlight/styles/atelier-seaside-light.css.p → src/css/highlight/atelier-seaside-light.css
View File

0
src/js/highlight/styles/atelier-sulphurpool-dark.css.p → src/css/highlight/atelier-sulphurpool-dark.css
View File

0
src/js/highlight/styles/atelier-sulphurpool-light.css.p → src/css/highlight/atelier-sulphurpool-light.css
View File

2
src/js/highlight/styles/atom-one-dark-reasonable.css.p → src/css/highlight/atom-one-dark-reasonable.css
View File

@ -9,10 +9,8 @@ Original One Dark Syntax theme from https://github.com/atom/one-dark-syntax
display: block;
overflow-x: auto;
padding: 0.5em;
line-height: 1.3em;
color: #abb2bf;
background: #282c34;
border-radius: 5px;
}
.hljs-keyword, .hljs-operator {
color: #F92672;

0
src/js/highlight/styles/atom-one-dark.css.p → src/css/highlight/atom-one-dark.css
View File

0
src/js/highlight/styles/atom-one-light.css.p → src/css/highlight/atom-one-light.css
View File

0
src/js/highlight/styles/brown-paper.css.p → src/css/highlight/brown-paper.css
View File

0
src/js/highlight/styles/brown-papersq.png → src/css/highlight/brown-papersq.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

0
src/js/highlight/styles/codepen-embed.css.p → src/css/highlight/codepen-embed.css
View File

0
src/js/highlight/styles/color-brewer.css.p → src/css/highlight/color-brewer.css
View File

3
src/js/highlight/styles/darcula.css.p → src/css/highlight/darcula.css
View File

@ -10,9 +10,6 @@ Darcula color scheme from the JetBrains family of IDEs
overflow-x: auto;
padding: 0.5em;
background: #2b2b2b;
}
.hljs {
color: #bababa;
}

0
src/js/highlight/styles/dark.css.p → src/css/highlight/dark.css
View File

0
src/js/highlight/styles/default.css.p → src/css/highlight/default.css
View File

0
src/js/highlight/styles/docco.css.p → src/css/highlight/docco.css
View File

0
src/js/highlight/styles/dracula.css.p → src/css/highlight/dracula.css
View File

0
src/js/highlight/styles/far.css.p → src/css/highlight/far.css
View File

3
src/js/highlight/styles/foundation.css.p → src/css/highlight/foundation.css vendored
View File

@ -10,7 +10,8 @@ Date: 2013-04-02
display: block;
overflow-x: auto;
padding: 0.5em;
background: #eee; color: black;
background: #eee;
color: black;
}
.hljs-link,

14
src/js/highlight/styles/github-gist.css.p → src/css/highlight/github-gist.css
View File

@ -1,5 +1,6 @@
/**
* GitHub Gist Theme
* Author : Anthony Attard - https://github.com/AnthonyAttard
* Author : Louis Barranqueiro - https://github.com/LouisBarranqueiro
*/
@ -16,7 +17,6 @@
color: #969896;
}
.hljs-string,
.hljs-variable,
.hljs-template-variable,
.hljs-strong,
@ -28,7 +28,7 @@
.hljs-keyword,
.hljs-selector-tag,
.hljs-type {
color: #a71d5d;
color: #d73a49;
}
.hljs-literal,
@ -53,7 +53,7 @@
.hljs-selector-class,
.hljs-selector-attr,
.hljs-selector-pseudo {
color: #795da3;
color: #6f42c1;
}
.hljs-addition {
@ -69,3 +69,11 @@
.hljs-link {
text-decoration: underline;
}
.hljs-number {
color: #005cc5;
}
.hljs-string {
color: #032f62;
}

0
src/js/highlight/styles/github.css.p → src/css/highlight/github.css
View File

2
src/js/highlight/styles/gml.css.p → src/css/highlight/gml.css
View File

@ -12,7 +12,7 @@ GML Theme - Meseta <meseta@gmail.com>
color: #C0C0C0;
}
.hljs-keywords {
.hljs-keyword {
color: #FFB871;
font-weight: bold;
}

0
src/js/highlight/styles/googlecode.css.p → src/css/highlight/googlecode.css
View File

122
src/css/highlight/gradient-dark.css Normal file
View File

@ -0,0 +1,122 @@
/*
Gradient Dark (c) Samia Ali <samiaab1990@gmail.com>
*/
.hljs
{
display: block;
overflow-x: auto;
padding: 0.5em;
background: rgb(80,31,122);
background: linear-gradient(166deg, rgba(80,31,122,1) 0%, rgba(40,32,179,1) 80%);
color:#e7e4eb;
}
.hljs-subtr{
color:#e7e4eb;
}
.hljs-doctag,
.hljs-meta,
.hljs-comment,
.hljs-quote
{
color:#af8dd9;
}
.hljs-selector-tag,
.hljs-selector-id,
.hljs-template-tag,
.hljs-regexp,
.hljs-attr,
.hljs-tag
{
color:#AEFBFF;
}
.hljs-params,
.hljs-selector-class,
.hljs-bullet
{
color:#F19FFF;
}
.hljs-keyword,
.hljs-section,
.hljs-meta-keyword,
.hljs-symbol,
.hljs-type
{
color:#17fc95;
}
.hljs-addition,
.hljs-number,
.hljs-link
{
color:#C5FE00;
}
.hljs-string
{
color: #38c0ff;
}
.hljs-attribute,
.hljs-addition
{
color:#E7FF9F;
}
.hljs-variable,
.hljs-template-variable
{
color:#E447FF;
}
.hljs-builtin-name,
.hljs-built_in,
.hljs-formula,
.hljs-name,
.hljs-title,
.hljs-class,
.hljs-function
{
color: #FFC800;
}
.hljs-selector-pseudo,
.hljs-deletion,
.hljs-literal
{
color:#FF9E44;
}
.hljs-emphasis,
.hljs-quote
{
font-style:italic;
}
.hljs-params,
.hljs-selector-class,
.hljs-strong,
.hljs-selector-tag,
.hljs-selector-id,
.hljs-template-tag,
.hljs-section,
.hljs-keyword
{
font-weight:bold;
}

130
src/css/highlight/gradient-light.css Normal file
View File

@ -0,0 +1,130 @@
/*
Gradient Light (c) Samia Ali <samiaab1990@gmail.com>
*/
.hljs
{
display: block;
overflow-x: auto;
padding: 0.5em;
background: rgb(255,253,141);
background: linear-gradient(142deg, rgba(255,253,141,1) 0%, rgba(252,183,255,1) 35%, rgba(144,236,255,1) 100%);
color:#250482;
}
.hljs-subtr{
color:#01958B;
}
.hljs-doctag,
.hljs-meta,
.hljs-comment,
.hljs-quote
{
color:#CB7200;
}
.hljs-selector-tag,
.hljs-selector-id,
.hljs-template-tag,
.hljs-regexp,
.hljs-attr,
.hljs-tag
{
color:#07BD5F;
}
.hljs-params,
.hljs-selector-class,
.hljs-bullet
{
color:#43449F;
}
.hljs-keyword,
.hljs-section,
.hljs-meta-keyword,
.hljs-symbol,
.hljs-type
{
color:#7D2801;
}
.hljs-addition,
.hljs-number,
.hljs-link
{
color:#7F0096;
}
.hljs-string
{
color: #38c0ff;
}
.hljs-attribute,
.hljs-addition
{
color:#296562;
}
.hljs-variable,
.hljs-template-variable
{
color:#025C8F;
}
.hljs-builtin-name,
.hljs-built_in,
.hljs-formula,
.hljs-name,
.hljs-title,
.hljs-class,
.hljs-function
{
color: #529117;
}
.hljs-selector-pseudo,
.hljs-deletion,
.hljs-literal
{
color:#AD13FF;
}
.hljs-emphasis,
.hljs-quote
{
font-style:italic;
}
.hljs-params,
.hljs-selector-class,
.hljs-strong,
.hljs-selector-tag,
.hljs-selector-id,
.hljs-template-tag,
.hljs-section,
.hljs-keyword
{
font-weight:bold;
}

0
src/js/highlight/styles/grayscale.css.p → src/css/highlight/grayscale.css
View File

0
src/js/highlight/styles/gruvbox-dark.css.p → src/css/highlight/gruvbox-dark.css
View File

0
src/js/highlight/styles/gruvbox-light.css.p → src/css/highlight/gruvbox-light.css
View File

1
src/js/highlight/styles/hopscotch.css.p → src/css/highlight/hopscotch.css
View File

@ -69,6 +69,7 @@
.hljs {
display: block;
overflow-x: auto;
background: #322931;
color: #b9b5b8;
padding: 0.5em;

0
src/js/highlight/styles/hybrid.css.p → src/css/highlight/hybrid.css
View File

0
src/js/highlight/styles/idea.css.p → src/css/highlight/idea.css
View File

0
src/js/highlight/styles/ir-black.css.p → src/css/highlight/ir-black.css
View File

0
src/js/highlight/styles/isbl-editor-dark.css.p → src/css/highlight/isbl-editor-dark.css
View File

3
src/js/highlight/styles/isbl-editor-light.css.p → src/css/highlight/isbl-editor-light.css
View File

@ -14,9 +14,8 @@ ISBL Editor style light color schemec (c) Dmitriy Tarasov <dimatar@gmail.com>
/* Base color: saturation 0; */
.hljs,
.hljs-subst {
color: #000000;
color: black;
}
.hljs-comment {

0
src/js/highlight/styles/kimbie.dark.css.p → src/css/highlight/kimbie.dark.css
View File

0
src/js/highlight/styles/kimbie.light.css.p → src/css/highlight/kimbie.light.css
View File

1
src/js/highlight/styles/lightfair.css.p → src/css/highlight/lightfair.css
View File

@ -8,6 +8,7 @@ Lightfair style (c) Tristian Kelly <tristian.kelly560@gmail.com>
display: block;
overflow-x: auto;
padding: 0.5em;
background: #fff;
}
.hljs-name {

88
src/css/highlight/lioshi.css Normal file
View File

@ -0,0 +1,88 @@
/* lioshi Theme */
/* Original theme - https://github.com/lioshi/vscode-lioshi-theme */
/* Comment */
.hljs-comment {
color: #8d8d8d;
}
/* quote */
.hljs-quote {
color: #b3c7d8;
}
/* Red */
.hljs-variable,
.hljs-template-variable,
.hljs-tag,
.hljs-name,
.hljs-selector-id,
.hljs-selector-class,
.hljs-regexp,
.hljs-deletion {
color: #cc6666;
}
/* Orange */
.hljs-number,
.hljs-built_in,
.hljs-builtin-name,
.hljs-literal,
.hljs-type,
.hljs-subst
.hljs-link {
color: #de935f;
}
/* Yellow */
.hljs-attribute {
color: #f0c674;
}
/* Green */
.hljs-string,
.hljs-bullet,
.hljs-params,
.hljs-addition {
color: #b5bd68;
}
/* Blue */
.hljs-title,
.hljs-meta,
.hljs-section {
color: #81a2be;
}
/* Purple */
.hljs-selector-tag,
.hljs-keyword,
.hljs-function,
.hljs-class {
color: #be94bb;
}
/* Purple light */
.hljs-symbol {
color: #dbc4d9;
}
.hljs {
display: block;
overflow-x: auto;
background: #303030;
color: #c5c8c6;
padding: 0.5em;
}
.hljs-emphasis {
font-style: italic;
}
.hljs-strong {
font-weight: bold;
}

2
src/js/highlight/styles/magula.css.p → src/css/highlight/magula.css
View File

@ -12,9 +12,9 @@ Music: Aphex Twin / Xtal
overflow-x: auto;
padding: 0.5em;
background-color: #f4f4f4;
color: black;
}
.hljs,
.hljs-subst {
color: black;
}

3
src/js/highlight/styles/mono-blue.css.p → src/css/highlight/mono-blue.css
View File

@ -6,9 +6,6 @@
overflow-x: auto;
padding: 0.5em;
background: #eaeef3;
}
.hljs {
color: #00193a;
}

0
src/js/highlight/styles/monokai-sublime.css.p → src/css/highlight/monokai-sublime.css
View File

3
src/js/highlight/styles/monokai.css.p → src/css/highlight/monokai.css
View File

@ -6,7 +6,8 @@ Monokai style - ported by Luigi Maselli - http://grigio.org
display: block;
overflow-x: auto;
padding: 0.5em;
background: #272822; color: #ddd;
background: #272822;
color: #ddd;
}
.hljs-tag,

182
src/css/highlight/night-owl.css Normal file
View File

@ -0,0 +1,182 @@
/*
Night Owl for highlight.js (c) Carl Baxter <carl@cbax.tech>
An adaptation of Sarah Drasner's Night Owl VS Code Theme
https://github.com/sdras/night-owl-vscode-theme
Copyright (c) 2018 Sarah Drasner
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
.hljs {
display: block;
overflow-x: auto;
padding: 0.5em;
background: #011627;
color: #d6deeb;
}
/* General Purpose */
.hljs-keyword {
color: #c792ea;
font-style: italic;
}
.hljs-built_in {
color: #addb67;
font-style: italic;
}
.hljs-type {
color: #82aaff;
}
.hljs-literal {
color: #ff5874;
}
.hljs-number {
color: #F78C6C;
}
.hljs-regexp {
color: #5ca7e4;
}
.hljs-string {
color: #ecc48d;
}
.hljs-subst {
color: #d3423e;
}
.hljs-symbol {
color: #82aaff;
}
.hljs-class {
color: #ffcb8b;
}
.hljs-function {
color: #82AAFF;
}
.hljs-title {
color: #DCDCAA;
font-style: italic;
}
.hljs-params {
color: #7fdbca;
}
/* Meta */
.hljs-comment {
color: #637777;
font-style: italic;
}
.hljs-doctag {
color: #7fdbca;
}
.hljs-meta {
color: #82aaff;
}
.hljs-meta-keyword {
color: #82aaff;
}
.hljs-meta-string {
color: #ecc48d;
}
/* Tags, attributes, config */
.hljs-section {
color: #82b1ff;
}
.hljs-tag,
.hljs-name,
.hljs-builtin-name {
color: #7fdbca;
}
.hljs-attr {
color: #7fdbca;
}
.hljs-attribute {
color: #80cbc4;
}
.hljs-variable {
color: #addb67;
}
/* Markup */
.hljs-bullet {
color: #d9f5dd;
}
.hljs-code {
color: #80CBC4;
}
.hljs-emphasis {
color: #c792ea;
font-style: italic;
}
.hljs-strong {
color: #addb67;
font-weight: bold;
}
.hljs-formula {
color: #c792ea;
}
.hljs-link {
color: #ff869a;
}
.hljs-quote {
color: #697098;
font-style: italic;
}
/* CSS */
.hljs-selector-tag {
color: #ff6363;
}
.hljs-selector-id {
color: #fad430;
}
.hljs-selector-class {
color: #addb67;
font-style: italic;
}
.hljs-selector-attr,
.hljs-selector-pseudo {
color: #c792ea;
font-style: italic;
}
/* Templates */
.hljs-template-tag {
color: #c792ea;
}
.hljs-template-variable {
color: #addb67;
}
/* diff */
.hljs-addition {
color: #addb67ff;
font-style: italic;
}
.hljs-deletion {
color: #EF535090;
font-style: italic;
}

106
src/css/highlight/nnfx-dark.css Normal file
View File

@ -0,0 +1,106 @@
/**
* nnfx dark - a theme inspired by Netscape Navigator/Firefox
*
* @version 1.0.0
* @author (c) 2020 Jim Mason <jmason@ibinx.com>
* @license https://creativecommons.org/licenses/by-sa/4.0 CC BY-SA 4.0
*/
.hljs {
display: block;
overflow-x: auto;
padding: 0.5em;
background: #333;
color: #fff;
}
.xml .hljs-meta {
font-weight: bold;
font-style: italic;
color: #69f;
}
.hljs-comment,
.hljs-quote {
font-style: italic;
color: #9c6;
}
.hljs-name,
.hljs-keyword {
color: #a7a;
}
.hljs-name,
.hljs-attr {
font-weight: bold;
}
.hljs-string {
font-weight: normal;
}
.hljs-variable,
.hljs-template-variable {
color: #588;
}
.hljs-code,
.hljs-string,
.hljs-meta-string,
.hljs-number,
.hljs-regexp,
.hljs-link {
color: #bce;
}
.hljs-title,
.hljs-symbol,
.hljs-bullet,
.hljs-built_in,
.hljs-builtin-name {
color: #d40;
}
.hljs-section,
.hljs-meta {
color: #a85;
}
.hljs-class .hljs-title,
.hljs-type {
color: #96c;
}
.hljs-function .hljs-title,
.hljs-attr,
.hljs-subst {
color: #fff;
}
.hljs-formula {
background-color: #eee;
font-style: italic;
}
.hljs-addition {
background-color: #797;
}
.hljs-deletion {
background-color: #c99;
}
.hljs-selector-id,
.hljs-selector-class {
color: #964;
}
.hljs-doctag,
.hljs-strong {
font-weight: bold;
}
.hljs-emphasis {
font-style: italic;
}

106
src/css/highlight/nnfx.css Normal file
View File

@ -0,0 +1,106 @@
/**
* nnfx - a theme inspired by Netscape Navigator/Firefox
*
* @version 1.0.0
* @author (c) 2020 Jim Mason <jmason@ibinx.com>
* @license https://creativecommons.org/licenses/by-sa/4.0 CC BY-SA 4.0
*/
.hljs {
display: block;
overflow-x: auto;
padding: 0.5em;
background: #fff;
color: #000;
}
.xml .hljs-meta {
font-weight: bold;
font-style: italic;
color: #48b;
}
.hljs-comment,
.hljs-quote {
font-style: italic;
color: #070;
}
.hljs-name,
.hljs-keyword {
color: #808;
}
.hljs-name,
.hljs-attr {
font-weight: bold;
}
.hljs-string {
font-weight: normal;
}
.hljs-variable,
.hljs-template-variable {
color: #477;
}
.hljs-code,
.hljs-string,
.hljs-meta-string,
.hljs-number,
.hljs-regexp,
.hljs-link {
color: #00f;
}
.hljs-title,
.hljs-symbol,
.hljs-bullet,
.hljs-built_in,
.hljs-builtin-name {
color: #f40;
}
.hljs-section,
.hljs-meta {
color: #642;
}
.hljs-class .hljs-title,
.hljs-type {
color: #639;
}
.hljs-function .hljs-title,
.hljs-attr,
.hljs-subst {
color: #000;
}
.hljs-formula {
background-color: #eee;
font-style: italic;
}
.hljs-addition {
background-color: #beb;
}
.hljs-deletion {
background-color: #fbb;
}
.hljs-selector-id,
.hljs-selector-class {
color: #964;
}
.hljs-doctag,
.hljs-strong {
font-weight: bold;
}
.hljs-emphasis {
font-style: italic;
}

0
src/js/highlight/styles/nord.css.p → src/css/highlight/nord.css
View File

Some files were not shown because too many files have changed in this diff Show More