Tag: Security

TLS Peer Verification w/PHP 5.6 and WordPress SMTP Email plugin

We ran into an issue after upgrading from PHP 5.5 to 5.6. We were no longer able to send email via the awesome WordPress SMTP Email plugin. Turns out that PHP 5.6 introduced some enhancements to its OpenSSL implementation. Stream wrappers now verify peer certificates and hostnames by default. This was causing the form submissions on our site to fail. Clearly there are some issues with our Postfix config and certs. While we sort out those issues, we were able to “solve” our immediate problem, and disabled peer verification in the plugin without editing core files. Thankfully the plugin developer included a filter that would allow us to modify the options array that is passed to PHPMailer.


Thanks random WordPress forum user for the solution!

What to do when your website is hacked/exploited

So your website has been “hacked”? You load your website in your browser, and are redirected to some spammers website, or maybe you google’d yourself (naughty), and found a few thousand indexed pages for knock off prada gear? Ok, so how do you fix this, and more importantly, how do you learn how they did it so you can defend against it later.


Secure the scene

The first thing I do is take a snapshot of the hacked web site. I want the entire webroot, and the access and error logs so that I can review them.

At this point I want to either shutdown the web server, or just restore the site and get back to the investigation. I would shut down if I felt the exploit was serious enough to warrant more serious action. Otherwise, just restore from backup, or checkout your repo, whatever you gotta do.

Here is where the fun begins. Try to figure out when, and how, your site was exploited. First things first, take an inventory of the scene.

Looking For Clues to how the site was hacked

Run the last command for suspicious logins. “last” will show a listing of last logged in users. Maybe someone SSH’d into your machine (hopefully not!) or logged in via FTP. If you see a suspicious IP in last then chances are someone has initiated the attack by simply uploading a file through FTP or SCP.

Check for modified files. If you are seeing files modified recently that you did not edit yourself they are suspect and probably contain malicious code. Check the modification timestamp on those files. That date will be useful when looking through the access logs. If you know that you have not made changes to your site in {n} days use that as the basis for your search:

Replace {n} with an actual number.

Scan all files in the webroot for common exploit signatures. If you find a lot of eval(gzinflate then you can almost be certain that it is malicious code. A good rule of thumb is eval is “evil”, except when it’s not (protip: it usually is).

Here is a bash script that checks for common exploits:

That will check all .php files under /var/www/vhosts for common php shells and exploits.

Collect Evidence

Ok, so you have a date, maybe a date range, and a list of potentially suspect files. With that information it’s time to start looking at logs. The goal is to be able to determine when, and potentially how, the attacker exploited the site.

There are 5 common vulnerabilities attackers exploit.

  1. Remote code execution
  2. SQL injection
  3. Format string vulnerabilities
  4. Cross Site Scripting (XSS)
  5. Username enumeration

All web developers should be familiar with the OWASP top 10 too. Anyways, you want to look for signs of those types of attacks. I scan my access logs and typically check for the following things:

  • Check for POST requests to suspicious locations, such as a file upload location, or other areas that are not accessed directly (like an includes directory, or theme directory).
  • Check for GET or POST requests with a remote URL as a parameter, such as /page/?url=http://atackersite.com/malicious_script
  • Check for GET or POST requests to the files you found when scanning the site for malicious code
  • Check for very odd query parameters in GET and POST requests
  • You should know your site, check for requests that are just out of the ordinary

When you find suspicious entries in your logs, take note of the IP addresses. Scan your logs for all entries for that IP, you might be able to find when the exploit occurred. If you do, you then know the requests that might have been the culprit, or at least have a date range for when the exploit took place. Check the logs around that date range. It could be that the attacker initiated the exploit through a proxy, and finished the work through another one.

Take Action

I recently had a clients website get exploited. The attacker modified dozens of PHP scripts, uploaded about 30 different PHP shells, and just generally made a mess of things. Most of the modified files had the same modification timestamp, so using that I checked the logs and found that someone had been crafting POSTs to a file upload plugin for a wysiwyg editor. Looking into that plugin, I found that they were able to upload a PHP shell. I quickly removed the plugin, restored the modified files, and removed all of the uploaded PHP shells.

The exploit showed me that the security of the website and server were severely lacking. SomeĀ  fundamental security precautions were not implemented. Such as allowing script execution in a user upload directory. That should just never happen. I setup some rules to disable script execution in specific directories, disable direct access to other directories, secured file and directory permissions.

I also setup fail2ban on the clients server. Fail2ban is great for monitoring logs and looking for certain signatures, and then banning the offending IP. You can define other actions to take, such as emailing you or something, but I just want to block their access to the server. So, ban via iptables. I noticed that this particular attacker was sending a lot of POST requests at the same time, so I setup a fail2ban filter to handle a postflood. You will probably want to setup filters for dealing with failed log in attempts to the server, and to address any of the other suspicious activity you found in your logs.

You will definitely want to update your CMS and update any 3rd party dependencies. Doing that alone will go a long way towards keeping your site exploit free.

Wrap up

Security is hard. It’s a never ending battle. And you will never, ever, make your site 100% safe and secure. The best thing to do is have a recovery plan. When your site is exploited can you recover? If you can answer yes to that, you are in a good place.

Securing Git repository from accidental exposure using Chef

It was brought to my attention at the office that a few of our recently launched websites had publicly exposed .git repository information. Unscrupulous users could use the exposed data to pull down the entire commit history, giving them unfiltered access to what is basically the blueprint for the website.

What if someone accidentally uploaded a config file to the repository with sensitive information in it? Or what if the user was able to discover a major security vulnerability in the code that would have otherwise remained “safe”? Scary.

There is definitely too much risk in allowing public access to your .git directory.

I created a Chef recipe called
gitsec.rb to deploy a very easy and quick fix to 20 web servers.

I was not sure the best way to determine if a service exists using Chef, so I just check for the appropriate init script. All of our servers are RHEL or CentOS, so I do not do any detection of platform here. It would be trivial to do so, so I will leave that to you!

You will also want to create a couple of templates:



At first I was returning a 403 status code, but realized that it was still announcing that a file did exist at that location. 404 is better, it does not expose the existence of a file.

This recipe is part of my initial web server setup now.