Christian Heilmann

Quick tip: using flatMap() to extract data from a huge set without any loop

Friday, September 6th, 2024 at 12:47 pm

A capybara wearing a flat cap and holding a pint with the name Flat Cap crossed out and .flatMap() instead.

I just created a massive dataset of all the AI generated metadata of the videos of the WeAreDeveloper World Congress and I wanted to extract only the tags.

The dataset is a huge array with each item containing a description, generated title, an array of tags, the original and their title, like this:

{
  "description": "The talk begins with an introduction to Twilio…", 
  "generatedtitle: "Enhancing Developer Experience: Strategies and Importance",
  "tags": ["Twilio", "DeveloperExperience", "CognitiveJourney"],
  "title": "Diving into Developer Experience"
}

What I wanted was an alphabetical lost of all the tags in the whole dataset, and this is a one-liner if you use flatMap():

data.flatMap(d => d.tags);

You can sort them alphabetically with sort():

data.flatMap(d => d.tags).sort();

And you can de-dupe the data and only get unique tags when you use Set():

new Set(data.flatMap(d => d.tags).sort());

You can try this in this codepen.

Share on Mastodon (needs instance)

Share on Twitter

Newsletter

Check out the Dev Digest Newsletter I write every week for WeAreDevelopers. Latest issues:

Dev Digest 146: 🥱 React fatigue 📊 Query anything with SQL 🧠 AI News

Why it may not be needed to learn React, why Deepfake masks will be a big problem and your spirit animal in body fat! 

Dev Digest 147: Free Copilot! Panel: AI and devs! RTO is bad! Pi plays!

Free Copilot! Experts discuss what AI means for devs. Don't trust containers. Mandated RTO means brain drain. And Pi plays Pokemon!

Dev Digest 148: Behind the scenes of Dev Digest & end of the year reports.

In 50 editions of Dev Digest we gave you 2081 resources. Join us in looking back and learn about all the trends this year.

Dev Digest 149: Wordpress break, VW tracking leak, ChatGPT vs Google.

Slowly starting 2025 we look at ChatGPT vs Google, Copilot vs. Cursor and the state of AI crawlers to replace web search…

Dev Digest 150: Shifting manually to AI.

Manual coding is becoming less of a skill. How can we ensure the quality of generated code? Also, unpacking an APK can get you an AI model.

My other work: