In addition to best-in-class Drupal training courses (pun intended), we at Debug Academy provide Drupal development services and Drupal 7 migration services. Our clients may self-host, host with Acquia, Pantheon, or even host with us.
Recently, a client who hosts their Drupal 9 website on Acquia reached out to to ask us to investigate an alert they had received from Acquia. Acquia sent an email which said "This email is to inform you that we emergency upsized your storage from 200GB to 300GB FS in Case #[redacted]. The cost to keep this upsize in place [is..]"
Acquia Emergency Upsize Alert
The emergency upsize alert is sent when one or more of your applications runs out of space. In this case, instead of allowing your application to 'break', Acquia may temporarily increase the storage space available to your application. They may or may do this at no cost to you temporarily, but if you decide to keep the "emergency upside" in-place without addressing the root cause, your costs are likely to increase.
So how do you "address the root cause" in this case..? First, you should understand the source of the problem.
Understanding Acquia's storage
On Acquia, there are multiple "pools" / types of storage that could run out of space.
Types of storage on Acquia hosting
At a high-level, there is typically at least two types of storage:
- File Storage
- Database Storage
You can view either one by:
- Logging in to Acquia
- Clicking on your application
- Clicking on an environment (such as Prod)
- Clicking "Stack metrics"
- Clicking either the "File System" tab (for File storage) or "Database" tab (for Database Storage)
- Scroll down, view the Storage used (e.g. 105GB out of 200GB)
Understanding different storage "pools" on Acquia
The "prod" environment gets its own isolated storage for both the File Storage and Database Storage. This means when you review the available storage for the prod environment, you're looking only at prod's available storage.
A common mistake is that people stop looking when they reach this point.
In fact, Acquia typically has a separate storage allocation for the non-prod environments. So your dev and stage environments will typically share storage space.
In practice, this ends up looking something like this:
- File storage
- Prod has its own file storage allocation (e.g. ~105GB out of 200GB used)
- The other environments (dev, stage, etc) share the "non-prod" file storage (e.g. 195GB out of 200GB used)
- Checking the storage space for any non-prod environment will typically show the same values as the rest of the non-prod environments, because they share storage space.
- Database storage
- Prod has its own database storage allocation (e.g. 20GB out of 100GB used)
- The other environments (dev, stage, etc) share the "non-prod" database storage (e.g. 30GB out of 100GB used)
- Again, checking the storage space for any non-prod environment will typically show the same values as the rest of the non-prod environments, because they share storage space.
Analyzing storage limits
When investigating storage space issues, you should typically look at all of the following:
- File Storage on prod over a period of time
- File Storage on dev over a period of time
- Database Storage on prod over a period of time
- Database Storage on dev over a period of time
You can view a chart of storage used over a period of time as follows:
- Click (e.g.) prod environment > Stack metrics >
- Click the date dropdown near the top of the page.
- Change it from a 1 day range to a longer range (e.g. 2 months) to view trends
- Do the same thing for the dev environment
- After choosing a date range, click the "File System" tab
- View the "Storage % used" chart
- In this example, I found the following chart which shows that the File system storage % used hit 100% in early January before dropping. It increased again in April before eventually leveling out.
- In this example, I found the following chart which shows that the File system storage % used hit 100% in early January before dropping. It increased again in April before eventually leveling out.
- View the "Storage % used" chart
- Do the same for the "File system" and "Database" tabs prod and dev environments, both the "File system" and "Database" tabs
The initial drop from 100% utilization was caused by Acquia increasing the allocated storage space for the environments. But is that a long-term solution?
Long-term solution for running out of file storage
Acquia temporarily upsized the storage space. Is that an appropriate long-term solution? As any experienced architect knows, the answer is always "it depends"!
There's nothing wrong with increasing the storage space (aside from incurring additional costs). However, we should first determine whether we actually need to be using the storage space. Here's how to investigate it further (warning: connecting via SSH is very powerful and very dangerous. Proceed with caution, and consider hiring a Drupal expert to investigate for you):
- In the Acquia UI, visit the environment (e.g. dev) which has the storage space problems
- Get the SSH information from the environment and use it to connect via SSH
- Check how much space is being used by each environment's files directory:
du -sh /mnt/gfs/*
- Review the output list. It will typically look something like this:
1.1M home
88G debugsclientdev
0 debugsclient.dev
98G debugsclientstg
0 debugsclient.test
1.1G debugsclientpredev
0 debugsclient.predev
In the example above, we can deduce that the "dev" site's files folder is using 88GB of space, while the "stage" site's files folder is using 98GB of space. That's a lot of space for two lower environments to be utilizing!
In this case, it seems the developers of the "debugsclient" website most likely copied all files from "prod" to "stage" environment, and again to the "dev" environment.
Saving space on Acquia by sharing files across environments
Instead of physically duplicating all files from prod to multiple lower environments, did you know you can "share" files across environments instead of duplicating them? There are a few ways this could be accomplished, but one popular solution is the "Stage files proxy" module:
https://www.drupal.org/project/stage_file_proxy
This module allows e.g. the "dev" site to load files from e.g. "prod" when the files can't be found on the dev server. That means you don't need to place the dev files on the dev server at all! Instead, skip copying the files from prod to dev altogether, and let stage file proxy fetch the files on-demand directly from prod - no more duplicated storage!
In this instance, we can save almost 200GB of storage space on the lower environments with this solution, and save ourselves from an unnecessary cost increase. Thanks, Acquia, for the emergency upsize - but our client doesn't need a permanent account upgrade. They can use the extra savings on Drupal courses (or an extravagant outing, if you're into that sort of thing).
Drupal expertise with a partner you can trust: Debug Academy
If you'd like us to be your Drupal partners, get in touch about our Drupal development services! We are a small team of experts who devote our attention to a small number of clients - don't wait!
There's more to the picture and we would love to share it with you. Experience the most efficient and effective path to a successful career in web development with Debug Academy. We have free Drupal training to help you get a sense of what’s possible in this field along with a more advanced curriculum. Learn about our Drupal certification training, Drupal migration services, and all the other Drupal courses we have to offer today!
Our approach blends structured learning with hands-on, real-world projects, guided by experienced professionals. Whether you're a beginner or looking to upgrade your skills, our comprehensive curriculum covers everything from fundamental coding principles to advanced web technologies.
With personalized mentorship, collaborative projects, and a supportive community, Debug Academy is committed to transforming you into a proficient, job-ready web developer.