Christian Heilmann

Quick tip: using flatMap() to extract data from a huge set without any loop

Friday, September 6th, 2024 at 12:47 pm

A capybara wearing a flat cap and holding a pint with the name Flat Cap crossed out and .flatMap() instead.

I just created a massive dataset of all the AI generated metadata of the videos of the WeAreDeveloper World Congress and I wanted to extract only the tags.

The dataset is a huge array with each item containing a description, generated title, an array of tags, the original and their title, like this:

{
  "description": "The talk begins with an introduction to Twilio…", 
  "generatedtitle: "Enhancing Developer Experience: Strategies and Importance",
  "tags": ["Twilio", "DeveloperExperience", "CognitiveJourney"],
  "title": "Diving into Developer Experience"
}

What I wanted was an alphabetical lost of all the tags in the whole dataset, and this is a one-liner if you use flatMap():

data.flatMap(d => d.tags);

You can sort them alphabetically with sort():

data.flatMap(d => d.tags).sort();

And you can de-dupe the data and only get unique tags when you use Set():

new Set(data.flatMap(d => d.tags).sort());

You can try this in this codepen.

Share on Mastodon (needs instance)

Share on BlueSky

Newsletter

Check out the Dev Digest Newsletter I write every week for WeAreDevelopers. Latest issues:

Don't stop thinking, AI Slop vs. OSS Security, rolling your own S3 Despite AI you still need to think, Bitter lessons from building AI products,  AI Slop vs. OSS security and pointer pointer…
200: Building for the web, what's left after rm -rf & 🌊🐴 vs AI What remains after you do a rm -rf? Why do LLMs know about a seahorse emoji? What image formats should you use? How private is your car?
Word is Doomed, Flawed LLM benchmarks, hard sorting and CSS mistakes Spot LLM benchmark flaws, learn why sorting is hard, how to run Doom in Word and how to say "no" like a manager.
30 years of JS, Browser AI, how attackers use GenAI, whistling code Learn how to use AI in your browser and not on the cloud, why AI makes different mistakes than humans and go and whistle up some code!
197: Dunning-Kruger steroids, state of cloud security, puppies>beer

My other work: