close
Skip to content

fix: escape character references in autolink destinations - #4053

Open
Kjubikstronk wants to merge 6 commits into
markedjs:masterfrom
Kjubikstronk:fix-link-href-entity-escaping
Open

fix: escape character references in autolink destinations#4053
Kjubikstronk wants to merge 6 commits into
markedjs:masterfrom
Kjubikstronk:fix-link-href-entity-escaping

Conversation

@Kjubikstronk

Copy link
Copy Markdown

Fixes #4052.

Following up on my comment there — the one-line change the issue suggests regresses inline links, so this takes the narrower route instead.

Why it is autolink-specific

CommonMark resolves character references in link destinations but not in autolinks ("backslash-escapes and entity references do not work in autolinks"). marked never resolves them anywhere — token.href keeps the literal source text — and that produces two different outcomes:

  • Inline link [t](…?x=1&lt;2) — the destination value should be ?x=1<2, which is written back into an attribute as &lt;. The unresolved source text is already exactly that, so the current output is right.
  • Autolink <…?x=1&lt;2> — the destination value is the literal text ?x=1&lt;2, which has to be written as &amp;lt;. Emitting it raw lets the browser decode it, so the link points at ?x=1<2.

That is why escaping every destination breaks the first case while fixing the second:

Input before escape everything this PR
<https://example.com/?x=1&lt;2> ?x=1&lt;2 ?x=1&amp;lt;2 ?x=1&amp;lt;2
[t](https://example.com/?a=1&amp;b=2) ?a=1&amp;b=2 ?a=1&amp;amp;b=2 ?a=1&amp;b=2

The change

Tokenizer.autolink() and Tokenizer.url() now set autolink: true on the token, and Renderer.link() escapes the destination and the text with escapeHtmlEntities(…, true) when it is set. The flag is an optional addition to Tokens.Link, so existing custom renderers and extensions are unaffected.

The text needed it too, not just the href — for <…?x=1&lt;2> the anchor text was also rendering as &lt;, because the default text escaping deliberately skips anything that already looks like a reference. Both halves now match the expected output in the issue.

This also fixes CommonMark example 595

Example 595 is in test/specs/commonmark with shouldFail: false, so it is expected to pass — but marked's actual output for it was:

<a href="https://foo.bar.baz/test?q=hello&id=22&boolean">…&amp;id=22&amp;boolean</a>

against an expected href of …?q=hello&amp;id=22&amp;boolean. The href and the link text disagreed with each other.

It was reported as passing because the spec runner compares with HtmlDiffer, which parses both sides — and in an attribute, &id=22 and &amp;id=22 parse to the same value. That normalisation hides the difference whenever the text is not a valid reference. It only becomes visible with something like &lt;, where the two parse to different values, which is what the issue reporter demonstrated with parse5.

So the spec suite could not have caught this, and still cannot. I've written the new tests as exact string comparisons in test/unit/marked.test.js rather than adding spec fixtures, for that reason.

I have deliberately not touched the unresolved-reference behaviour itself — examples 32 and 33 are still marked shouldFail, and [foo](/f&ouml;&ouml;) still renders unchanged. That is the larger gap tracked in #4050 and wants its own discussion.

Verification

  • test:specs — 1779 pass, 0 fail, unchanged before and after.
  • test:unit — 189 pass, up from 185; the 5 exec failures are pre-existing on main and fail identically with this change stashed.
  • tsc --noEmit and eslint clean on the touched files.
  • Removing the source change makes 3 of the 4 new tests fail; the fourth is the inline-link guard, which passes either way by design.

The four Lexer token assertions for autolinks and urls were updated for the new field.

Character references are not resolved inside an autolink, so its
destination and text are literal, but both were written into the output
unescaped. A destination containing a valid reference such as `&lt;` was
decoded again by the browser, so the link pointed somewhere other than
what was written.

Mark autolink and extended url tokens and escape every `&` when
rendering them. Inline links keep the source text, which is already
valid in an attribute, so they are unchanged.
@vercel

vercel Bot commented Aug 15, 2026

Copy link
Copy Markdown

@Kjubikstronk is attempting to deploy a commit to the MarkedJS Team on Vercel.

A member of the Team first needs to authorize it.

Comment thread test/unit/marked.test.js Outdated
@vercel

vercel Bot commented Aug 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
Image marked-website Ready Ready Preview Aug 19, 2026 4:30am

Request Review

@UziTech

UziTech commented Aug 16, 2026

Copy link
Copy Markdown
Member

Looks like this also doesn't follow CommonMark for

[example](http://example.com?foo=1&bar=2)

CommonMark demo

<p><a href="http://example.com?foo=1&amp;bar=2">example</a></p>

@Kjubikstronk

Copy link
Copy Markdown
Author

You're right, and my reasoning in the description was too narrow. I checked that a destination containing a reference round-trips correctly and concluded inline destinations were fine, but a bare & is a different case and it was still being written out raw.

The two need different treatment, and escapeHtmlEntities already has both modes:

  • inline destination — the text is kept as written, so &lt; is already right in an attribute and must not be touched, while &bar=2 cannot begin a reference and has to become &amp;bar=2. That is the default mode.
  • autolink — references are not resolved, so every & is literal and all of them are escaped. That is the encode mode.

Which is a one-line change now that the autolink flag exists:

href = escapeHtmlEntities(cleanHref, autolink);

Image sources had the same gap, so they are escaped too.

[example](http://example.com?foo=1&bar=2)   ->  href="http://example.com?foo=1&amp;bar=2"
![i](http://example.com?foo=1&bar=2)        ->  src="http://example.com?foo=1&amp;bar=2"
[t](http://example.com?a=1&lt;2)            ->  href="http://example.com?a=1&lt;2"        (unchanged)
[t](http://example.com?a=1&amp;b=2)         ->  href="http://example.com?a=1&amp;b=2"     (unchanged)
<https://example.com/?x=1&lt;2>             ->  href="https://example.com/?x=1&amp;lt;2"

Tests added for the bare-ampersand case in both a link and an image, plus one pinning that a reference in an inline destination is left alone.

test:specs 1779 pass / 0 fail, unchanged. test:unit 192 pass, up from 189; the 5 exec failures are pre-existing on main and fail identically with this change stashed.

An inline link destination keeps its source text, so a reference such as
`&lt;` is already correct in an attribute, but a bare `&` is not. Escape
the ones that cannot begin a reference, in image sources as well, and
keep escaping everything inside an autolink.
<p><a href="https://example.com/?a=1&amp;b=2">t</a></p>
<p><a href="http://example.com?foo=1&amp;bar=2">example</a></p>
<p><img src="http://example.com?foo=1&amp;bar=2" alt="i"></p>
<p><a href="http://example.com?a=1&lt;2">t</a></p>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to this CommonMark demo this line should be

<p><a href="http://example.com?a=1%3C2">t</a></p>

@UziTech UziTech Aug 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use these as the tests:

---
renderExact: true
---
https://example.com/?x=1&lt;2

https://example.com/?y=1&amp;2

https://example.com/?a=1&b=2

<https://example.com/?x=1&lt;2>

<https://example.com/?y=1&amp;2>

<https://example.com/?a=1&b=2>

[https://example.com/?x=1&lt;2](https://example.com/?x=1&lt;2)

[https://example.com/?y=1&amp;2](https://example.com/?y=1&amp;2)

[https://example.com/?a=1&b=2](https://example.com/?a=1&b=2)

[https://example.com/?x=1&lt;2][link1]

[https://example.com/?y=1&amp;2][link2]

[https://example.com/?a=1&b=2][link3]

![https://example.com/?x=1&lt;2](https://example.com/?x=1&lt;2)

![https://example.com/?y=1&amp;2](https://example.com/?y=1&amp;2)

![https://example.com/?a=1&b=2](https://example.com/?a=1&b=2)

[link1]: https://example.com/?x=1&lt;2
[link2]: https://example.com/?y=1&amp;2
[link3]: https://example.com/?a=1&b=2
<p><a href="https://example.com/?x=1&amp;lt;2">https://example.com/?x=1&amp;lt;2</a></p>
<p><a href="https://example.com/?y=1&amp;amp;2">https://example.com/?y=1&amp;amp;2</a></p>
<p><a href="https://example.com/?a=1&amp;b=2">https://example.com/?a=1&amp;b=2</a></p>
<p><a href="https://example.com/?x=1&amp;lt;2">https://example.com/?x=1&amp;lt;2</a></p>
<p><a href="https://example.com/?y=1&amp;amp;2">https://example.com/?y=1&amp;amp;2</a></p>
<p><a href="https://example.com/?a=1&amp;b=2">https://example.com/?a=1&amp;b=2</a></p>
<p><a href="https://example.com/?x=1%3C2">https://example.com/?x=1&lt;2</a></p>
<p><a href="https://example.com/?y=1&amp;2">https://example.com/?y=1&amp;2</a></p>
<p><a href="https://example.com/?a=1&amp;b=2">https://example.com/?a=1&amp;b=2</a></p>
<p><a href="https://example.com/?x=1%3C2">https://example.com/?x=1&lt;2</a></p>
<p><a href="https://example.com/?y=1&amp;2">https://example.com/?y=1&amp;2</a></p>
<p><a href="https://example.com/?a=1&amp;b=2">https://example.com/?a=1&amp;b=2</a></p>
<p><img src="https://example.com/?x=1%3C2" alt="https://example.com/?x=1&lt;2"></p>
<p><img src="https://example.com/?y=1&amp;2" alt="https://example.com/?y=1&amp;2"></p>
<p><img src="https://example.com/?a=1&amp;b=2" alt="https://example.com/?a=1&amp;b=2"></p>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the tests to add reflinks and show the difference between the href and text of some

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swapped in your cases, thanks — the reflinks and the href/text split make the autolink-vs-inline difference much clearer than what I had.

On the %3C one: you're right that CommonMark gives http://example.com?a=1%3C2 there, but that line comes out the same on master as on this branch. I diffed the whole fixture both ways — this PR only changes the six autolink lines and the ?a=1&b=2 cases, where master emits a raw & in the href. The three &lt; inline/reference/image destinations are byte-identical before and after.

So the missing percent-encoding looks like a separate gap rather than something I've introduced. Happy to take it on, either here or as its own PR — just say which you'd prefer, since it's a different code path from the entity escaping.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment in the file about the lines that don't match CommonMark

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added it to the frontmatter — front-matter runs the block through YAML, so # comments are stripped and never reach the rendered output. The fixture still passes byte for byte.

It calls out that the &lt; inline links, reference links and images are the ones that differ: CommonMark resolves the reference and percent-encodes it to ?x=1%3C2 where marked keeps ?x=1&lt;2. I noted that this comes from URL encoding rather than from the escaping in this PR, and that it is unchanged on master, so nobody reads those lines later as something this PR introduced.

@UziTech

UziTech commented Aug 18, 2026

Copy link
Copy Markdown
Member

Interestingly GitHub doesn't parse this one correctly

<https://example.com/?y=1&amp;2>

CommonMark

<p><a href="https://example.com/?y=1&amp;amp;2">https://example.com/?y=1&amp;amp;2</a></p>

GitHub https://example.com/?y=1&2

<p><a href="https://example.com/?y=1&amp;2">https://example.com/?y=1&amp;2</a></p>

@UziTech UziTech left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

& not HTML-escaped in link/autolink href, silently changing the link target

2 participants