Christian Heilmann

A code snippet to scrape all headings and their target URLs from a markdown generated page

Friday, February 19th, 2021 at 7:54 pm

When you use Markdown to write your documentation most static page generators will generate IDs for each of the headings in the document to allow you to navigate directly to them.

Markdown: 
## Gerbils and other rodents
HTML: 
<h2 id="gerbils-and-other-rodents">Gerbils and other Rodents</h2>

To go directly there you can then use https://example.com#gerbils-and-other-rodents if you published at example.com.

The other day I was asked to create a list of all the links in the What’s new in Devtools 89 document, which is generated from Markdown. The list should be the text of the headline followed by the full URL to get to that part of the document. This was to batch generate some shortURLs from them.

I am pretty sure there are a lot of clever ways to do that by scraping but as I like my browser environment, I just used the Console to do that. Here’s the script that you can paste into the Console:

let out = '';
$$(':is(h1,h2,h3,h4,h5,h6)[id]').forEach(elm => {
   out += `${elm.innerText}
${document.location.href}#${elm.id}
` 
});
copy(out);

You can see it in action in the following GIF:

Running the script in the console to get all the heading information

The next step was to store this as a Snippet and next time I just need to run that.

Share on Mastodon (needs instance)

Share on BlueSky

Newsletter

Check out the Dev Digest Newsletter I write every week for WeAreDevelopers. Latest issues:

Don't stop thinking, AI Slop vs. OSS Security, rolling your own S3 Despite AI you still need to think, Bitter lessons from building AI products,  AI Slop vs. OSS security and pointer pointer…
200: Building for the web, what's left after rm -rf & 🌊🐴 vs AI What remains after you do a rm -rf? Why do LLMs know about a seahorse emoji? What image formats should you use? How private is your car?
Word is Doomed, Flawed LLM benchmarks, hard sorting and CSS mistakes Spot LLM benchmark flaws, learn why sorting is hard, how to run Doom in Word and how to say "no" like a manager.
30 years of JS, Browser AI, how attackers use GenAI, whistling code Learn how to use AI in your browser and not on the cloud, why AI makes different mistakes than humans and go and whistle up some code!
197: Dunning-Kruger steroids, state of cloud security, puppies>beer

My other work: