👋 Hello there

Today's article is all about how you can make a Random Meme API using Node.js and web scraping. We'll be using Fastify for our API and we'll be using Puppeteer to scrape the web and get the random meme.

This random meme API was inspired by the same kind of API here. But I wanted to build it using Node.js and Puppeteer.

We'll be scraping Memedroid using the Puppeteer NPM package.

Initializing the project

First of all, we'll need to create a folder with the name we want and we'll need to initialize our Node.js app in that folder. We do it using this command:

npm init -y

Then we need two dependencies to install, simply install puppeteer and fastify.

npm i puppeteer fastify

Using Fastify for API

After we have our project set up and our dependencies installed, we're good to go with writing the code! Create an index.js file and import fastify with this code to create the server.

Once this thing is done, when we run the app using node index, our app will be running on port 5555. But let's create the base route (/ )for it.

Getting the random meme using Puppeteer

Here comes the fun part now! We'll open the web browser and get all the images from memedroid, and we'll do all of it through code.

With the puppeteer package, Chromium also comes installed to scrape the web. That's why it might have taken time for you to get installed.

To skip Chromium download, you can use puppeteer-core package and add the path to your Chrome file following the docs. Or go through this article.

We'll create a function to get all memes and then we'll pick a random one in the route.

We simply launch the browser and open the page for memedroid in this code 👆.

Now let's get all the <img> tags which are in the <div> with the class of item-aux-container. That's where all the memes live in.

As in the above image, inside of each <article> tag, the div with that class exists, so we simply get it using the $$eval method on the page.

This method takes two arguments:

Selector
Callback function with the element(s)

We will map over the images in the callback function, and we'll return only the URL of the image from getting the src attribute. And this is how we do it.

We check if the src attribute starts with http and ends with jpeg and we return that if it does.

Unfortunately, that also returns to us null if that's not the case, so we filter out the nulls using the .filter() method.

Once all that work is done, we close the browser and return the array of images, this is how the whole function looks like:

Using Fastify to send the random meme

Finally, we will pick a random meme and send it to the user using this code:

Now, whenever the user visits localhost:5555, they get this:

We have our app done! Thanks for reading! You can find all the code here.

I hope you liked it! Comment down your thoughts! There is always room for improvement so let me know your suggestions on this project!

Connect with me on my YouTube channel and my Twitter 😉

Until next time, keeping awesome ✌️