Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

👀 Safe preview: Open notebooks without running #2563

Merged
merged 56 commits into from
Oct 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
3cb1b25
👀 Open notebooks without running
fonsp May 15, 2023
2808105
sanitize html WIP
fonsp May 17, 2023
c08a968
Merge branch 'main' into wait-to-run
fonsp May 17, 2023
4639b6e
merge part 2
fonsp May 17, 2023
0d1594c
Use DOMPurify instead of Sanitize API
fonsp May 18, 2023
f2d4523
Link HTML sanitization to JL permission
fonsp May 18, 2023
483b04d
disable fancy html features when sanitized
fonsp May 18, 2023
3115e28
Prevent "restart required" when modifying environment before starting…
fonsp May 18, 2023
6ac699b
Show risky URL source
fonsp Sep 18, 2023
3d6637b
Merge branch 'main' into wait-to-run
fonsp Sep 18, 2023
6fa2501
Store "risky source" in notebook metadata
fonsp Sep 18, 2023
82d089f
Fix "Start notebook in background" button
fonsp Sep 18, 2023
ce84ff7
shorter warning
fonsp Sep 18, 2023
091b48f
Update DOMPurify.d.ts
fonsp Sep 19, 2023
5265e83
css tweaks
fonsp Sep 19, 2023
d919bd3
popup: big option
fonsp Sep 19, 2023
b234b5a
Better info about Safe preview mode
fonsp Sep 19, 2023
3c434a6
Configuration `warn_about_untrusted_code` to disable warnings
fonsp Sep 19, 2023
03d033e
derp
fonsp Sep 19, 2023
ff3a890
Clearly show code and unexecuted cells for untrusted notebook
fonsp Sep 19, 2023
002e8de
css tweaks
fonsp Sep 19, 2023
3be70e7
Update editor.css
fonsp Sep 19, 2023
d10f789
Merge branch 'main' into wait-to-run
fonsp Sep 19, 2023
0e936c2
style tweakz
fonsp Sep 19, 2023
67f8f4c
Render md-only cells in safe preview mode
fonsp Sep 19, 2023
bdcc0d5
small screen CSS tweak
fonsp Sep 19, 2023
afd4a37
Merge branch 'main' into wait-to-run
fonsp Oct 21, 2023
4106b5a
click permission button in frontend tests
fonsp Oct 21, 2023
3436dc8
fixture
fonsp Oct 21, 2023
329d430
Merge branch 'main' into wait-to-run
fonsp Oct 21, 2023
66f7041
Fix sanitization in log display
fonsp Oct 21, 2023
0868a91
backend tests
fonsp Oct 21, 2023
929e9e3
Safe preview for notebooks from clipboard
fonsp Oct 21, 2023
b46d910
part 2
fonsp Oct 21, 2023
1414f65
Frontend tests for opening files
fonsp Oct 21, 2023
1ddabcd
Show message about sanitized html
fonsp Oct 21, 2023
518ad34
Some frontend tests
fonsp Oct 21, 2023
2fa997f
more frontend test stability
fonsp Oct 21, 2023
643a4db
Render cells as queued when restarting process
fonsp Oct 21, 2023
c6340cb
Merge branch 'main' into wait-to-run
fonsp Oct 21, 2023
dbffdea
fix
fonsp Oct 22, 2023
ad2035d
Merge branch 'main' into wait-to-run
fonsp Oct 22, 2023
99ff590
test modifying cells
fonsp Oct 22, 2023
084578b
Merge branch 'main' into wait-to-run
fonsp Oct 22, 2023
a3f5072
Update slide_controls.js
fonsp Oct 22, 2023
fe1adbf
Merge branch 'main' into wait-to-run
fonsp Oct 22, 2023
90dcedc
ugh
fonsp Oct 22, 2023
a4b85cb
Keep the prompt because you are used to press enter
fonsp Oct 22, 2023
ab575e6
asdf
fonsp Oct 22, 2023
87a5a2f
Merge branch 'main' into wait-to-run
fonsp Oct 22, 2023
ca4cc95
fixxx
fonsp Oct 22, 2023
1187e09
Test risky URL
fonsp Oct 22, 2023
2e770f1
hoppa
fonsp Oct 22, 2023
2296d94
frontend tests
fonsp Oct 22, 2023
da27294
tweak
fonsp Oct 22, 2023
bc5860c
Make HTML santizer message more stable
fonsp Oct 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions frontend/common/Binder.js
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ export const start_binder = async ({ setStatePromise, connect, launch_params })
const upload_url = with_token(
with_query_params(new URL("notebookupload", binder_session_url), {
name: new URLSearchParams(window.location.search).get("name"),
execution_allowed: "true",
})
)
console.log(`downloading locally and uploading `, upload_url, launch_params.notebookfile)
Expand Down
7 changes: 7 additions & 0 deletions frontend/common/ProcessStatus.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export const ProcessStatus = {
ready: "ready",
starting: "starting",
no_process: "no_process",
waiting_to_restart: "waiting_to_restart",
waiting_for_permission: "waiting_for_permission",
}
1 change: 1 addition & 0 deletions frontend/common/RunLocal.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ export const start_local = async ({ setStatePromise, connect, launch_params }) =
with_query_params(new URL("notebookupload", binder_session_url), {
name: new URLSearchParams(window.location.search).get("name"),
clear_frontmatter: "yesplease",
execution_allowed: "yepperz",
})
),
{
Expand Down
31 changes: 24 additions & 7 deletions frontend/components/Cell.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { RunArea, useDebouncedTruth } from "./RunArea.js"
import { cl } from "../common/ClassTable.js"
import { PlutoActionsContext } from "../common/PlutoContext.js"
import { open_pluto_popup } from "./Popup.js"
import { SafePreviewOutput } from "./SafePreviewUI.js"

const useCellApi = (node_ref, published_object_keys, pluto_actions) => {
const [cell_api_ready, set_cell_api_ready] = useState(false)
Expand Down Expand Up @@ -96,6 +97,8 @@ const on_jump = (hasBarrier, pluto_actions, cell_id) => () => {
* selected: boolean,
* force_hide_input: boolean,
* focus_after_creation: boolean,
* process_waiting_for_permission: boolean,
* sanitize_html: boolean,
* [key: string]: any,
* }} props
* */
Expand All @@ -110,6 +113,8 @@ export const Cell = ({
focus_after_creation,
is_process_ready,
disable_input,
process_waiting_for_permission,
sanitize_html = true,
nbpkg,
global_definition_locations,
}) => {
Expand All @@ -134,8 +139,8 @@ export const Cell = ({

const remount = useMemo(() => () => setKey(key + 1))
// cm_forced_focus is null, except when a line needs to be highlighted because it is part of a stack trace
const [cm_forced_focus, set_cm_forced_focus] = useState(/** @type{any} */ (null))
const [cm_highlighted_range, set_cm_highlighted_range] = useState(null)
const [cm_forced_focus, set_cm_forced_focus] = useState(/** @type {any} */ (null))
const [cm_highlighted_range, set_cm_highlighted_range] = useState(/** @type {{from, to}?} */ (null))
const [cm_highlighted_line, set_cm_highlighted_line] = useState(null)
const [cm_diagnostics, set_cm_diagnostics] = useState([])

Expand Down Expand Up @@ -200,9 +205,11 @@ export const Cell = ({

const class_code_differs = code !== (cell_input_local?.code ?? code)
const class_code_folded = code_folded && cm_forced_focus == null
const no_output_yet = (output?.last_run_timestamp ?? 0) === 0
const code_not_trusted_yet = process_waiting_for_permission && no_output_yet

// during the initial page load, force_hide_input === true, so that cell outputs render fast, and codemirrors are loaded after
let show_input = !force_hide_input && (errored || class_code_differs || !class_code_folded)
let show_input = !force_hide_input && (code_not_trusted_yet || errored || class_code_differs || !class_code_folded)

const [line_heights, set_line_heights] = useState([15])
const node_ref = useRef(null)
Expand Down Expand Up @@ -291,6 +298,7 @@ export const Cell = ({
show_input,
shrunk: Object.values(logs).length > 0,
hooked_up: output?.has_pluto_hook_features ?? false,
no_output_yet,
})}
id=${cell_id}
>
Expand All @@ -310,7 +318,11 @@ export const Cell = ({
>
<span></span>
</button>
${cell_api_ready ? html`<${CellOutput} errored=${errored} ...${output} cell_id=${cell_id} />` : html``}
${code_not_trusted_yet
? html`<${SafePreviewOutput} />`
: cell_api_ready
? html`<${CellOutput} errored=${errored} ...${output} sanitize_html=${sanitize_html} cell_id=${cell_id} />`
: html``}
<${CellInput}
local_code=${cell_input_local?.code ?? code}
remote_code=${code}
Expand Down Expand Up @@ -342,7 +354,12 @@ export const Cell = ({
onerror=${remount}
/>
${show_logs && cell_api_ready
? html`<${Logs} logs=${Object.values(logs)} line_heights=${line_heights} set_cm_highlighted_line=${set_cm_highlighted_line} />`
? html`<${Logs}
logs=${Object.values(logs)}
line_heights=${line_heights}
set_cm_highlighted_line=${set_cm_highlighted_line}
sanitize_html=${sanitize_html}
/>`
: null}
<${RunArea}
cell_id=${cell_id}
Expand Down Expand Up @@ -409,15 +426,15 @@ export const Cell = ({
* [key: string]: any,
* }} props
* */
export const IsolatedCell = ({ cell_input: { cell_id, metadata }, cell_result: { logs, output, published_object_keys }, hidden }) => {
export const IsolatedCell = ({ cell_input: { cell_id, metadata }, cell_result: { logs, output, published_object_keys }, hidden }, sanitize_html = true) => {
const node_ref = useRef(null)
let pluto_actions = useContext(PlutoActionsContext)
const cell_api_ready = useCellApi(node_ref, published_object_keys, pluto_actions)
const { show_logs } = metadata

return html`
<pluto-cell ref=${node_ref} id=${cell_id} class=${hidden ? "hidden-cell" : "isolated-cell"}>
${cell_api_ready ? html`<${CellOutput} ...${output} cell_id=${cell_id} />` : html``}
${cell_api_ready ? html`<${CellOutput} ...${output} sanitize_html=${sanitize_html} cell_id=${cell_id} />` : html``}
${show_logs ? html`<${Logs} logs=${Object.values(logs)} line_heights=${[15]} set_cm_highlighted_line=${() => {}} />` : null}
</pluto-cell>
`
Expand Down
50 changes: 38 additions & 12 deletions frontend/components/CellOutput.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { html, Component, useRef, useLayoutEffect, useContext } from "../imports/Preact.js"

import DOMPurify from "../imports/DOMPurify.js"

import { ErrorMessage, ParseError } from "./ErrorMessage.js"
import { TreeView, TableView, DivElement } from "./TreeView.js"

Expand All @@ -24,6 +26,7 @@ import { pluto_syntax_colors, ENABLE_CM_MIXED_PARSER } from "./CellInput.js"
import hljs from "../imports/highlightjs.js"
import { julia_mixed } from "./CellInput/mixedParsers.js"
import { julia_andrey } from "../imports/CodemirrorPlutoSetup.js"
import { SafePreviewSanitizeMessage } from "./SafePreviewUI.js"

export class CellOutput extends Component {
constructor() {
Expand All @@ -50,8 +53,8 @@ export class CellOutput extends Component {
})
}

shouldComponentUpdate({ last_run_timestamp }) {
return last_run_timestamp !== this.props.last_run_timestamp
shouldComponentUpdate({ last_run_timestamp, sanitize_html }) {
return last_run_timestamp !== this.props.last_run_timestamp || sanitize_html !== this.props.sanitize_html
}

componentDidMount() {
Expand Down Expand Up @@ -113,7 +116,7 @@ export let PlutoImage = ({ body, mime }) => {
return html`<img ref=${imgref} type=${mime} src=${""} />`
}

export const OutputBody = ({ mime, body, cell_id, persist_js_state = false, last_run_timestamp }) => {
export const OutputBody = ({ mime, body, cell_id, persist_js_state = false, last_run_timestamp, sanitize_html = true }) => {
switch (mime) {
case "image/png":
case "image/jpg":
Expand All @@ -130,23 +133,24 @@ export const OutputBody = ({ mime, body, cell_id, persist_js_state = false, last
// NOTE: Jupyter doesn't do this, jupyter renders everything directly in pages DOM.
// -DRAL
if (body.startsWith("<!DOCTYPE") || body.startsWith("<html")) {
return html`<${IframeContainer} body=${body} />`
return sanitize_html ? null : html`<${IframeContainer} body=${body} />`
} else {
return html`<${RawHTMLContainer}
cell_id=${cell_id}
body=${body}
persist_js_state=${persist_js_state}
last_run_timestamp=${last_run_timestamp}
sanitize_html=${sanitize_html}
/>`
}
break
case "application/vnd.pluto.tree+object":
return html`<div>
<${TreeView} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} />
<${TreeView} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} />
</div>`
break
case "application/vnd.pluto.table+object":
return html`<${TableView} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} />`
return html`<${TableView} cell_id=${cell_id} body=${body} persist_js_state=${persist_js_state} sanitize_html=${sanitize_html} />`
break
case "application/vnd.pluto.parseerror+object":
return html`<div><${ParseError} cell_id=${cell_id} ...${body} /></div>`
Expand All @@ -155,7 +159,7 @@ export const OutputBody = ({ mime, body, cell_id, persist_js_state = false, last
return html`<div><${ErrorMessage} cell_id=${cell_id} ...${body} /></div>`
break
case "application/vnd.pluto.divelement+object":
return DivElement({ cell_id, ...body, persist_js_state })
return DivElement({ cell_id, ...body, persist_js_state, sanitize_html })
break
case "text/plain":
if (body) {
Expand All @@ -177,7 +181,7 @@ export const OutputBody = ({ mime, body, cell_id, persist_js_state = false, last
}
}

register(OutputBody, "pluto-display", ["mime", "body", "cell_id", "persist_js_state", "last_run_timestamp"])
register(OutputBody, "pluto-display", ["mime", "body", "cell_id", "persist_js_state", "last_run_timestamp", "sanitize_html"])

let IframeContainer = ({ body }) => {
let iframeref = useRef()
Expand Down Expand Up @@ -469,7 +473,7 @@ let declarative_shadow_dom_polyfill = (template) => {
}
}

export let RawHTMLContainer = ({ body, className = "", persist_js_state = false, last_run_timestamp }) => {
export let RawHTMLContainer = ({ body, className = "", persist_js_state = false, last_run_timestamp, sanitize_html = true }) => {
let pluto_actions = useContext(PlutoActionsContext)
let pluto_bonds = useContext(PlutoBondsContext)
let js_init_set = useContext(PlutoJSInitializingContext)
Expand All @@ -481,7 +485,7 @@ export let RawHTMLContainer = ({ body, className = "", persist_js_state = false,

useLayoutEffect(() => {
if (container_ref.current && pluto_bonds) set_bound_elements_to_their_value(container_ref.current.querySelectorAll("bond"), pluto_bonds)
}, [body, persist_js_state, pluto_actions, pluto_bonds])
}, [body, persist_js_state, pluto_actions, pluto_bonds, sanitize_html])

useLayoutEffect(() => {
const container = container_ref.current
Expand All @@ -498,8 +502,30 @@ export let RawHTMLContainer = ({ body, className = "", persist_js_state = false,
// @ts-ignore
dump.append(...container.childNodes)

let html_content_to_set = sanitize_html
? DOMPurify.sanitize(body, {
FORBID_TAGS: ["style"],
})
: body

// Actually "load" the html
container.innerHTML = body
container.innerHTML = html_content_to_set

if (html_content_to_set !== body) {
// DOMPurify also resolves HTML entities, which can give a false positive. To fix this, we use DOMParser to parse both strings, and we compare the innerHTML of the resulting documents.
const parser = new DOMParser()
const p1 = parser.parseFromString(body, "text/html")
const p2 = parser.parseFromString(html_content_to_set, "text/html")

if (p2.documentElement.innerHTML !== p1.documentElement.innerHTML) {
console.info("HTML sanitized", { body, html_content_to_set })
let info_element = document.createElement("div")
info_element.innerHTML = SafePreviewSanitizeMessage
container.prepend(info_element)
}
}

if (sanitize_html) return

let scripts_in_shadowroots = Array.from(container.querySelectorAll("template[shadowroot]")).flatMap((template) => {
// @ts-ignore
Expand Down Expand Up @@ -564,7 +590,7 @@ export let RawHTMLContainer = ({ body, className = "", persist_js_state = false,
js_init_set?.delete(container)
invalidate_scripts.current?.()
}
}, [body, persist_js_state, last_run_timestamp, pluto_actions])
}, [body, persist_js_state, last_run_timestamp, pluto_actions, sanitize_html])

return html`<div class="raw-html-wrapper ${className}" ref=${container_ref}></div>`
}
Expand Down
Loading