cft

😱 Make a Random Meme API With Node.js and Puppeteer

Build a Random Meme API using Node.js and Puppeteer by scraping memedroid!


user

Usman Sabuwala

2 years ago | 2 min read

👋 Hello there

Today's article is all about how you can make a Random Meme API using Node.js and web scraping. We'll be using Fastify for our API and we'll be using Puppeteer to scrape the web and get the random meme.

This random meme API was inspired by the same kind of API here. But I wanted to build it using Node.js and Puppeteer.

We'll be scraping Memedroid using the Puppeteer NPM package.


Initializing the project

First of all, we'll need to create a folder with the name we want and we'll need to initialize our Node.js app in that folder. We do it using this command:

npm init -y

Then we need two dependencies to install, simply install puppeteer and fastify.

npm i puppeteer fastify

Using Fastify for API

After we have our project set up and our dependencies installed, we're good to go with writing the code! Create an index.js file and import fastify with this code to create the server.

Server in Fastify
Server in Fastify

Once this thing is done, when we run the app using node index, our app will be running on port 5555. But let's create the base route (/ )for it.

Handling get request on / route
Handling get request on / route

Getting the random meme using Puppeteer

Here comes the fun part now! We'll open the web browser and get all the images from memedroid, and we'll do all of it through code.

With the puppeteer package, Chromium also comes installed to scrape the web. That's why it might have taken time for you to get installed.

To skip Chromium download, you can use puppeteer-core package and add the path to your Chrome file following the docs. Or go through this article.

We'll create a function to get all memes and then we'll pick a random one in the route.

We simply launch the browser and open the page for memedroid in this code 👆.
We simply launch the browser and open the page for memedroid in this code 👆.

Now let's get all the <img> tags which are in the <div> with the class of item-aux-container. That's where all the memes live in.

Meme location in HTML
Meme location in HTML

As in the above image, inside of each <article> tag, the div with that class exists, so we simply get it using the $$eval method on the page.

This method takes two arguments:

  1. Selector
  2. Callback function with the element(s)
Selecting all images on the page
Selecting all images on the page

We will map over the images in the callback function, and we'll return only the URL of the image from getting the src attribute. And this is how we do it.

We check if the src attribute starts with http and ends with jpeg and we return that if it does.

Returning appropriate images
Returning appropriate images

Unfortunately, that also returns to us null if that's not the case, so we filter out the nulls using the .filter() method.

Filtering null from the array
Filtering null from the array

Once all that work is done, we close the browser and return the array of images, this is how the whole function looks like:

Final getAllMemes function
Final getAllMemes function

Using Fastify to send the random meme

Finally, we will pick a random meme and send it to the user using this code:

Sending a response with random meme
Sending a response with random meme

Now, whenever the user visits localhost:5555, they get this:

Response from the API
Response from the API

We have our app done! Thanks for reading! You can find all the code here.

I hope you liked it! Comment down your thoughts! There is always room for improvement so let me know your suggestions on this project!

Connect with me on my YouTube channel and my Twitter 😉

Until next time, keeping awesome ✌️

Upvote


user
Created by

Usman Sabuwala


people
Post

Upvote

Downvote

Comment

Bookmark

Share


Related Articles