Christian Heilmann

Quick tip: Getting all links from any web site into a spreadsheet using browser developer tools

Thursday, August 24th, 2023 at 3:15 pm

As part of taking over the editorial job of the WeAreDevelopers newsletter, I needed to get all the links from older editions and import them into a spreadsheet. Eventually I will write a scraping script, but there is a much simpler way to get all the links from a web site into a spreadsheet: browser developer tools. You can follow how that’s done in this video:

The step-by-step instructions:

  • Go to the web site you want the links from
  • Open Developer Tools (press F12)
  • Go to the Console tool
  • Enter (or copy and paste) the following: console.table($$('a'),['innerHTML','href'])
  • Highlight the table with your mouse
  • Copy and paste into your spreadsheet, making sure to only copy the values

This works in Microsoft Edge, Google Chrome, Firefox but for some reason not in Safari

Another issue is that long links and link texts get shortened with a … in Chrome and Edge, to work around that you need to shorten the links first. In my case, I removed the `utm_source=` tracking info. To this end, I had to write another short script in the console to run first.

Using $$('a').forEach(l => {l.href = l.href.replace(/?utm_source=.*$/,'')}) replaces all the utm_source tracking info on each link before calling the console.table.

Share on Mastodon (needs instance)

Share on Twitter

Newsletter

Check out the Dev Digest Newsletter I write every week for WeAreDevelopers. Latest issues:

Dev Digest 146: 🥱 React fatigue 📊 Query anything with SQL 🧠 AI News

Why it may not be needed to learn React, why Deepfake masks will be a big problem and your spirit animal in body fat! 

Dev Digest 147: Free Copilot! Panel: AI and devs! RTO is bad! Pi plays!

Free Copilot! Experts discuss what AI means for devs. Don't trust containers. Mandated RTO means brain drain. And Pi plays Pokemon!

Dev Digest 148: Behind the scenes of Dev Digest & end of the year reports.

In 50 editions of Dev Digest we gave you 2081 resources. Join us in looking back and learn about all the trends this year.

Dev Digest 149: Wordpress break, VW tracking leak, ChatGPT vs Google.

Slowly starting 2025 we look at ChatGPT vs Google, Copilot vs. Cursor and the state of AI crawlers to replace web search…

Dev Digest 150: Shifting manually to AI.

Manual coding is becoming less of a skill. How can we ensure the quality of generated code? Also, unpacking an APK can get you an AI model.

My other work: