Tag: PHP

Profiling and Debugging a PHP app with Xdebug and Docker

I have started using an IDE again (PHPStorm) so that I could debug some applications and do some basic app profiling. I want to use Xdebug to profile my PHP apps. I am using Docker Compose on Windows 10. I have made this very complicated for myself but here we go.

The directory structure of my app looks like:

First thing is to get Xdebug setup in the PHP container.

I am using a custom Dockerfile for my PHP container where I install a ton of additional modules and packages, install wp-cli, and copy a custom php.ini to the container.

Here is the entire Dockerfile for the PHP container:

custom php.ini under ./build/docker/php/:

Some important things here. I am creating a directory to store Xdebug output (/tmp/xdebug) which will be used by another container to parse and display the output. In the custom php.ini we tell Xdebug to store its output to this directory. We also configure Xdebug to enable remote debugging so that we can debug from our IDE. If you do not want to debug EVERY request you should disable remote_autostart. If you do this you need to pass in a specific GET/POST parameter to trigger the debugger (typically XDEBUG_PROFILE). Make note of the remote_port and idekey values. We need these when we configure our IDE.

In your IDE you would configure Xdebug to listen on port 9999 for connections and to use the IDE Session Key MYIDE to ensure you are only debugging requests that use that session key (really only necessary for complicated setups with multiple apps on the same server).

There are two environment variables that I set on the PHP container that are required to make this all work.


XDEBUG_CONFIG is required to tell Xdebug where the client is running. To be honest, I am not sure if this is actually required or is only required because of PHPStorm. I am using Docker Toolbox and am using the IP from the VirtualBox VM where the Docker env is running. It would be great to not have to have this param as it would be more portable.

The variable PHP_IDE_CONFIG though is required for PHPStorm, and it tells my IDE which server configuration to use.

Neither of these may be required if you are using native docker and a different IDE.  /shrug

The first part of this is done. We can now debug an app from our IDE. The second thing I wanted to do was run a profiler and inspect the results. Xdebug will output cachegrind files. We just need a way to inspect them. There are some desktop apps you can use, like KCacheGrind, QCacheGrind, WinCacheGrind, etc. Your IDE may even be able to parse them (PHPStorm is currently no able to for some reason). Or you can use a web based system. I opted for a web based system using WebGrind. There is, conveniently, a docker container for this.

I configured the php container to expose /tmp/xdebug as a shared volume, which is where Xdebug is configured too output cachegrind files. Then I configured the webgrind container to mount that volume. Also I pass an environment variable to tell WebGrind where to find the cachegrind files:


With that we can go to and start digging into the app profiles.

Complete docker-compose.yml


WP Transients must be used responsibly

We ran into an interesting issue with WooCommerce at work. First, here is the subject of the support request we got from our hosting provider:

The site is generating ~150MB/sec of transaction logs, filling 500GB of diskspace

Holy. Shit. A WordPress site should not be generating that much data. 150MB per second? Wow.

How? Why?

The simple explanation is that there is a bottleneck in WooCommerce with the filtered layer nav query objects using single transient record.

What is happening here is that a sql query based on the currently selected filters is hashed and shoved into an array that is saved to a single transient record. This means that every single interaction with the filters requires a read and possible write to a single transient record. A site with any sort of traffic and let’s say 9 filter widgets (with around 50 total options) will potentially generate a huge amount of unique queries. It is no wonder why we are pushing 150MB/s.

Our quick, temporary, patch was to simply remove the transient.

You can see the massive improvement in performance after removing the transients. We applied the patch around 9:47 am.

Object caching would probably help. I was surprised at how much of an improvement we saw by simply removing the transient.

I think a good solution here would be to use unique transients for each hashed query, and not a single transient for EVERY hashed query. It would work find on small WP installs and would scale.

I will try it out and see what we get and if the results are good I will submit a PR to the woocommerce devs.


I said we should use transients responsibly. In this case, I would be creating potentially 15k additional (tiny) transient records. Is that more responsible than 1 massive 1mb transient?

WooCommerce devs has asked that I run some performance tests. Going to do so and report back!

update 2:

Not having any transients at all is better at scale in our case since the SQL query that is executed is not that heavy and we have some decent page caching via varnish. Also our MySQL server is well tuned. Every single request to a page with the layered nav will make N requests for the transient data. If data has to be written, that is N updates per request. This single record becomes a bottleneck as the field is locked while it is being written to. Redis or Memcache would be a a better solution. WP Transients are just bad on their own.

Order Posts via “Weighted Pseudo Randomness”

A request we are getting more often is to show a list of posts, to elevate some of those posts above others, and to show the posts in a random order. Imagine a post type called “Sponsors”. Sponsors are tiered, like “Platinum”, “Gold”, “Silver”, etc. We want the Platinum sponsors to appear before Gold, Gold before Silver, and so on. We don’t want to favor one particular Platinum sponsor though, we want them to be randomized but ordered by the tier.

The number one rule we had for implementing this feature is that we could not break existing WordPress functionality. Meaning, pagination has to work, and any sort of post filtering that existed on the site must also continue to work. An additional requirement is that we could not have duplicate posts show up as we paginated results. Hence, pseudo random.

This is achieved by ordering the result set by a weight value (aka the tier) and then to randomize those results, using the MySQL RAND() method. MySQL’s RAND() method takes an optional $SEED parameter. If we pass the same $SEED value to our query for each request we can maintain the same random order as we paginate through posts. This ensures that we do not have duplicates. I am generating the seed value using “date(‘ymd’)”. The value will change daily, and create a new randomness to the posts each day. Weight is derived from the tier that the posts are assigned. In my case, we use ACF and an ACF field that allows a user to select a single value from a Tier taxonomy. Knowing this, I used the postmeta value of the term id that is selected to get the slug of the term itself. I then used a CASE statement in my query to assign a weight value based on the slug of the selected taxonomy term. Tier1 is assigned 1, tier2 is assigned 2, if there is no term, weight is 9999 (so that these posts always show up after a tiered post). The CASE statement looks like:

In order for this to work we need to JOIN the wp_terms table based on the metavalue of the selected tier taxonomy.

The query basically looks like this when it is compiled (this is a simplified example of how the resulting MySQL query is going to look):

The goal is to make WordPress write this query for us in the loop. We can do this using filters that modify the WP_Query. The first thing we need to do is to be able to identify the WP_Query so that we do not alter _other_ queries on the site. We only want to change the query that loads posts from our custom Sponsors post type. To do this we add a custom query_var to WordPress and then check for that query_var in our wp_query. Add the query var:

We now inject this param into our main query using “pre_get_posts”. We do not want our listing of sponsor posts in the admin area of WordPress to be ordered randomly, so we need to check that we are not is_admin().

This function checks the wp_query object passed to it. If the post type is our custom post type we set the “is_pseudorandom_query” query var to true. With this set we can now setup our wp_query filters. If you are using a custom WP_Query object you can pass “is_pseudorandom_query” as one of the $args:

Now to the WP_Query filters. The three filters we need are posts_fields to add our CASE statement, posts_join to add our custom LEFT JOINS, and posts_orderby to order by our new weight value and then by RAND().

The functions:

In each function we inspect the passed $wp_query object to see if “is_pseudorandom_query” is set. If it is, then we modify the query.

And there it is. We can now order posts by tier, and then randomize each tier.

TLS Peer Verification w/PHP 5.6 and WordPress SMTP Email plugin

We ran into an issue after upgrading from PHP 5.5 to 5.6. We were no longer able to send email via the awesome WordPress SMTP Email plugin. Turns out that PHP 5.6 introduced some enhancements to its OpenSSL implementation. Stream wrappers now verify peer certificates and hostnames by default. This was causing the form submissions on our site to fail. Clearly there are some issues with our Postfix config and certs. While we sort out those issues, we were able to “solve” our immediate problem, and disabled peer verification in the plugin without editing core files. Thankfully the plugin developer included a filter that would allow us to modify the options array that is passed to PHPMailer.


Thanks random WordPress forum user for the solution!

Enable status for php-fpm

Accessing the PHP-FPM Status screen is easy enough. First, enable pm.status in your php pool:

Then add the following block to your Nginx vhost conf:

Restart php-fpm and nginx and then browse to http://<SERVERIP>/status. You will be presented with some useful information on the current status of your PHP-FPM pool.

By default /status will just show a short status, like an overview of all of the processes. To see output on each process append ?full to the url, http://<SERVERIP>/status?full. You can also pass ?json to get JSON output, if you wanted to feed the data into some other log or stats processing tool (thinking like, greylog or logstash?).

Here is a breakdown of the stats presented to you:

pool – the name of the pool.
process manager – static, dynamic or ondemand.
start time – the date and time FPM has started.
start since – number of seconds since FPM has started.
accepted conn – the number of request accepted by the pool.
listen queue – the number of request in the queue of pending connections (see backlog in listen(2)).
max listen queue – the maximum number of requests in the queue of pending connections since FPM has started.
listen queue len – the size of the socket queue of pending connections.
idle processes – the number of idle processes.
active processes – the number of active processes.
total processes – the number of idle + active processes.
max active processes – the maximum number of active processes since FPM has started.
max children reached – number of times the process limit has been reached.

Use this information to tune your pool configuration.

A WordPress ajax handler for custom themes

A WordPress ajax handler for custom themes

Something I have been noodling on is a better way to handle ajax requests in my custom themes. Seems to me that a relatively complex theme ends up with a lot of add_action calls for custom ajax handlers, and this could be simplified/reduced. Every time a new endpoint is required we have to add two new add_action calls to our theme. Maybe a better approach is to write a single ajax endpoint that will route requests to the proper classes/methods?

The goal would be that all ajax requests are run through our custom ajax handler and routed to the appropriate controller & method for execution. This method allows us to encapsulate functionality into separate classes/modules instead of cluttering functions.php with ajax functions. Potentially this makes the code more reusable.

With some sane configuration I think that this could be a good way to build a flexible ajax interface into your theme. To protect against malicious calls classes are namespaced (\Ecs\Modules\), and the methods are prefixed (ajax*). This should stop most attempts to execute arbitrary theme code. There are issues though. There would be a fatal error if the class does not exist which could expose information about the environment. We have to make sure that the $_REQUEST params are well sanitized. This would need to be scrutinized and tested for security issues. We don’t want someone to be able to craft a request to execute code we don’t explicitly want executed.

Here is an example structure in a hypothetical Theme class.