What I’ve Learned From Being Hacked
Take security seriously before it affects you
Yes, I was hacked. This is the kind of situation that you think can only happen to other people. But guess what? You are the other people’s other people and it might happen to you!
The moment I discovered it, I started to have a mix of feelings — like the five stages of grief. I was in a sort of parallel world, trying to escape reality and hoping the problem was never there in the first place or that it might resolve itself like magic.
That didn’t happen, of course. After a few hours spent finding a solution and figuring out what had happened, I learned some valuable lessons.
In this article, I want to share them with you.
I work for a small company that offers solutions in predictive maintenance and asset management. In a simple way, we connect different sensors’ data into a gateway and then send all the collected info to a cloud system where it is analyzed.
There’s a dashboard where you can visualize the data and manage the system. All data are stored on a MongoDB database and accessed via an API.
This API is the only way to access the database and each user has their own token generated by the API. All the connections are done using HTTPS.
A few months ago, we installed the system in a big company as a proof of concept so they could evaluate it and decide whether they would buy it or not. That represented a huge opportunity for us.
After two months of tests, everything was running smoothly. All sensors were collecting data on time.
But one day, one of my colleagues called me saying he could not access the dashboard. He was getting an “invalid password” error. At first, I thought it might be a mistake. Maybe the server was down or something like that.
When I accessed the server, however, I realized the problem was much more serious. All the data had been deleted. We had been hacked.
It was a strange moment. Confusing feelings started to grow inside me, like the five stages of grief:
- Denial: This could not have happened to me. Maybe a coworker changed something, set up a different server. A prank maybe. But then I remembered I was the only one who had access to the servers.
- Anger: With a lot of other companies out there, why us?
Photo by Camila Quintero Franco on Unsplash.
- Bargaining: Maybe we could afford to pay what they were demanding. It was a small amount anyway. Everything could go back to how it was before.
- Depression: I felt like I was not good enough for this job, I hadn’t learned anything in ten years of software development. I was a fraud. In reality, it was just impostor syndrome all over again.
- Acceptance: That was it. I was going to be fired and would never find another job in this field.
All these feelings lasted for about five minutes, but it felt like an eternity. Time had stopped.
Photo by Ben White on Unsplash.
Then the emotions settled, I calmed down, and started to figure out what the problem was and how to solve it.
Upon inspecting the database, all the tables and collections had been erased. A new table called
READ_ME_TO_RECOVER_YOUR_DATA was there.
Ransom note from attacker. Photo from Help Net Security.
After a little research on the internet, I learned this is a very common ransomware attack on MongoDB databases that has been happening since 2017. They attack databases that don’t have security enabled and are open to the world.
Just a couple of days before the incident, I disabled the firewall and opened the Mongo service to accept connections from any IP.
Why would I do that?
Because I didn’t want to enter my login credentials to get a backup of the data.
Photo by Holger Link on Unsplash.
That’s right. I was hacked because I was lazy. I’m not proud to admit it. I can say I even feel ashamed of that. But that is the truth.
After learning about the attack, we warned the client. We worked together to recover the data from our last backup (one day of data was lost), implement the official security checklist from MongoDB, update all the systems and services, and change all the passwords, along with other best practices for security.
Here are some lessons I’ve learned.
Don’t be lazy
Just to save five minutes, I could have lost a big contract and a lot of money. Don’t try to save time just because you don’t want to enter your password every time you have to do something or access your system.
Because of that laziness, I ended up working all weekend to solve a problem that should not have happened in the first place.
The first action you may consider in this situation is to cover up the problem. Don’t do that.
As soon as we found out what happened, we warned our client. It is really important to inform all the people who might be impacted by the situation so they can also act early and mitigate any problems that might appear.
By being honest, the client and other interested parties can trust that you are not hiding any information from them and that you are doing your best to protect their data.
Since we do not store any sensitive data in our database and all the architecture is isolated, we did not have any damages.
Constantly back up your data. I mean, do it right now. Create a script to automate it for you. Just do it!
If we did not have any backup, two months of data and hard work would have been lost.
Change the default configuration for any system you are using — especially for the admin user and their password. And keep it in a safe place.
Check for updates
Check for exploits and security updates often. Sites like CVE have a list with a lot of vulnerabilities in systems and programs. Check them often and make sure your system is up to date.
I hope my experience can help others to not make the same mistake I did. Only a brief moment of distraction can create an accident.
And this can happen to anyone. It doesn’t matter how experienced you are. Even though I have worked with security systems for quite some time, I made a mistake that could have been devastating.