Parsing HTML strings that contain script tags
Updated:
In my last post I experimented with some ways to implement HTML includes with the technology available in browsers today. In that post I created a Custom Element that would fetch the HTML from a URL and inject that into the DOM. One of the things you might expect to happen during this process is if the HTML contained a <script>
tag, it should immediately (or eventually if it was deferred or async) evaluate that script upon insertion. Turns out that not all ways of parsing HTML strings containing script tags will execute scripts, in fact there's only 1 method that will excute them out of the box.
These are the main methods that parse an HTML string into a DOM tree:
innerHTML
outerHTML
insertAdjacentHTML
DOMParser.parseFromString
Range.createContextualFragment
element.setHTMLUnsafe
Document.parseHTMLUnsafe
In the examples below, I'm going to include this piece of HTML, which can be found here, and you'll see in each demo whether it worked or not.
<div>
<p>
This is a test include. You should have seen an <code>alert</code> by now
</p>
<script>
window.alert("script ran");
</script>
</div>
innerHTML #
This applies to outerHTML
, insertAdjacentHTML
, setHTMLUnsafe
, and parseHTMLUnsafe
as well.
Here's a Custom Element that will fetch the HTML fragment and insert it using the Custom Element's innerHTML
:
<x-include src="/path/to/header.html"></x-include>
<script>
class CustomInclude extends HTMLElement {
constructor() {
super();
}
getSrc(src) {
fetch(src)
.then((response) => response.text())
.then((domString) => {
this.innerHTML = domString;
});
}
connectedCallback() {
const src = this.getAttribute("src");
this.getSrc(src);
}
}
window.customElements.define("x-include", CustomInclude);
</script>
Try it out:
This doesn't work because using innerHTML
disables scripts for XSS security reasons. That doesn't make innerHTML
safe, read more about security concerns here. But if you 100% trust the HTML source you're including, then you can get around this with some extra code:
class CustomInclude extends HTMLElement {
constructor() {
super();
}
getSrc(src) {
fetch(src)
.then((response) => response.text())
.then((domString) => {
const frag = document.createDocumentFragment();
// documentFragments don't have an innerHTML so we need to use
// an intermediate element to convert the domString to nodes
const tempDiv = document.createElement("div");
tempDiv.innerHTML = domString;
for (const child of tempDiv.childNodes) {
frag.appendChild(child);
}
let scriptOnlyProperties;
// scripts created using `createElement` will be evaluated
// upon insertion into the DOM so we just need to go through
// all of the scripts in the documentFragment and replace them
// with a new instance using document.createElement('script');
Array.from(frag.querySelectorAll("script")).forEach((oldScript) => {
const newScript = document.createElement("script");
if (!scriptOnlyProperties) {
// get only HTMLScriptElement properties, not the inherited ones
// from HTMLElement. We don't need the constructor either.
scriptOnlyProperties = Object.getOwnPropertyNames(
Object.getPrototypeOf(newScript)
).filter((prop) => prop !== "constructor");
}
for (const prop in scriptOnlyProperties) {
// one of the properties of a script element is .text which will
// be the code in the script if it has any
if (oldScript[prop]) newScript[prop] = oldScript[prop];
}
oldScript.parentElement.replaceChild(newScript, oldScript);
});
this.innerHTML = "";
this.appendChild(frag);
})
.catch((e) => {
console.warn(e);
});
}
connectedCallback() {
const src = this.getAttribute("src");
this.getSrc(src);
}
}
window.customElements.define("x-include", CustomInclude);
Using DOMParser #
Another way of parsing HTML strings is by using the DOMParser
API.
<x-include src="/path/to/header.html"></x-include>
<script>
class CustomInclude extends HTMLElement {
constructor() {
super();
}
parse(domString) {
const root = new DOMParser().parseFromString(
domString,
"text/html"
).documentElement;
const html = document.importNode(root, true);
const body = html.querySelector("body").childNodes;
this.innerHTML = "";
this.append(...body);
}
getSrc(src) {
fetch(src)
.then((response) => response.text())
.then((domString) => {
this.parse(domString);
});
}
connectedCallback() {
const src = this.getAttribute("src");
this.getSrc(src);
}
}
window.customElements.define("x-include", CustomInclude);
</script>
This doesn't work because the HTML5 spec states that scripts inside documents created by using DOMParser
should not be evaluated.
Again, you can probably get around this with the same method above where you replace the scripts with newly created script
tags.
Range.createContextualFragment #
I actually didn't know about this one. I learned about it while reading through the whatwg discussion I linked to in my previous post.
<x-include src="/path/to/header.html"></x-include>
<script>
class CustomInclude extends HTMLElement {
constructor() {
super();
}
parse(domString) {
const range = document.createRange();
range.selectNode(this);
const documentFragment = range.createContextualFragment(domString);
return documentFragment;
}
getSrc(src) {
fetch(src)
.then((response) => response.text())
.then((domString) => {
const domFrag = this.parse(domString, this);
this.innerHTML = "";
this.appendChild(domFrag);
})
.catch((e) => {
console.warn(e);
});
}
connectedCallback() {
const src = this.getAttribute("src");
this.getSrc(src);
}
}
window.customElements.define("x-include", CustomInclude);
</script>
Finally our JS got executed without any workarounds or hacks!
But why do some of these work and some don't?
When parsing HTML from a string and a script
tag is encountered, the HTML Parser will internally maintain a boolean state called "already started" that will indicate whether this script will be evaluated or not. Depending on which parsing alogrithm is used determines whether this is initially set to true or false. Only a value of false means that the script should be executed upon insertion.
innerHTML
, outerHTML
, insertAdjacentHTML
, element.setHTMLUnsafe
, and Document.parseHTMLUnsafe
all use the HTML fragment parsing algorithm which sets the "already started" flag to true, which means it should not be executed:
4. If the parser was created as part of the HTML fragment parsing algorithm, then set the script element's already started to true. (fragment case)
Source: https://html.spec.whatwg.org/multipage/parsing.html#scriptTag
As I mentioned above, the spec for DOMParser
states that scripts should not be evaluated:
Note that script elements are not evaluated during parsing, and the resulting document's encoding will always be UTF-8. The document's URL will be inherited from parser's relevant global object.
Source: 8.5.1 The DOMParser interface
The only method that does work is Range.createContextualFragment
because the spec explicitly states that scripts "already started" state should be set to false
, meaning that it should be executed upon insertion:
8. For each script of fragment node's script element descendants:
1. Set script's already started to false.
What I'm not sure about is why the working group decided that createContextualFragment
is the method that will execute scripts. Apparently it was originally an undocumented API that some libraries used (mainly PrototypeJS). I found some discussion about it here: https://bugzilla.mozilla.org/show_bug.cgi?id=599588. There was some disagreement and then they had an offline voice discussion and all agreed that scripts should be runnable. 🤷♂️
Update #
Fixed error and improved code in one of the code samples