Visit the site

Visit the site

Organizations Involved

Community contributions

For YMCA IT directors and managers: the open-source platform on which your website runs is now reviewed more rigorously. Bugs that would previously slip through β€” because no reviewer had 60 minutes to test a PR manually β€” now get caught before they reach any production site.

For organizations evaluating Drupal development partners: this is what infrastructure investment looks like in practice β€” not just writing code, but building the systems that make code quality sustainable at scale.

Overview

Organization: ITCare LLC β€” Drupal development and managed hosting for YMCA associations
Project: PR Builder β€” automated sandbox system for YCloudYUSA/yusaopeny
Timeline: March 7–11, 2026 (4 working days)
Result: 108 commits, 6 PRs merged, 3 live sandbox environments per pull request, ~4:30 build time

About the project

The Problem

YUSAOpenY is an open-source Drupal 11 distribution used by YMCA associations across North America. Contributors submit pull requests that modify Drupal modules, themes, and configuration. Before PR Builder existed, reviewing a pull request required:

  1. Clone the branch locally
  2. Set up a local Drupal environment (DDEV, Docker, or similar)
  3. Run composer install, configure Solr, import demo content
  4. Wait β€” often 30 to 60 minutes β€” before seeing any result

Most reviewers skipped this entirely. They read the code diff without testing the actual behavior. UI regressions went unnoticed. Database migration bugs (hook_update_N) slipped into releases that affected dozens of YMCA associations at once.

We decided to fix this.


What We Built

PR Builder is an automated sandbox system that deploys three live testing environments for every pull request submitted to the YUSAOpenY repository, and posts the URLs directly in the GitHub PR comment β€” automatically, within minutes of each push.

No setup. No waiting. Just click a link.

Three sandbox types for three real questions

Sandbox Question It Answers
Install Does a fresh drush site:install work with this PR's code?
Upgrade Do database migrations (hook_update_N) work when upgrading from the current stable release?
Code Code deployed only β€” for reviewers who want to run their own testing.

The Upgrade sandbox is especially valuable: it uses a pre-built base database (current stable release, extended preset) and runs drush updb against the PR's code. This catches migration bugs invisible on fresh installs.

Reviewers get a single GitHub comment β€” updated in place on every new commit β€” with direct links, status indicators (βœ… ready / ❌ failed), and downloadable SQLite database snapshots. The build pipeline runs in ~4 minutes 30 seconds end-to-end.


The Engineering Foundation: Making site:install Fit on a $24/Month Server

Before building PR Builder at all, we had a fundamental blocker: Drupal's site:install was too heavy to run on a budget server.

On Drupal 11.1, installing the full YUSAOpenY profile consumed 4.38 GB of RAM and took 537 seconds (almost 9 minutes). Our target server was a $24/month DigitalOcean droplet (2 CPU, 4 GB RAM). The math didn't work.

Andrii Podanenko (YUSAOpenY lead maintainer) diagnosed the root causes and contributed a series of upstream fixes:

PR #327 β€” Batch Module Installation (Drupal core issue #3416522)

Drupal rebuilds its entire service container after every single module install. With 60+ modules in the YUSAOpenY profile, that meant 60 container rebuilds β€” each one slower than the last as more services loaded. This is O(nΒ²) behavior.

Starting with Drupal 11.2, core supports batched container rebuilds. PR #327 rewrote the YUSAOpenY install profile to batch module installation in chunks of 20:

// Before: 60 container rebuilds
foreach ($modules as $module) {
    \Drupal::service('module_installer')->install([$module]);
}

// After: 3 container rebuilds for 60 modules
foreach (array_chunk($modules, 20) as $batch) {
    \Drupal::service('module_installer')->install($batch);
}

PR #329 β€” MenuTreeStorage Query Optimization (Drupal core issue #3493290)

During site:install, Drupal's MenuTreeStorage::doSave() queried the database individually for each menu link to check if it already exists. With hundreds of menu links being created, this generated 230,127 SQL queries. PR #329 added a core patch that batch-loads existing menu links at the start of rebuild() instead of querying one-by-one.

PR #331 β€” Route Rebuild Optimization (Drupal core issue #3492438)

Drupal rebuilt the entire routing table every time a module was installed β€” even in batch mode. PR #331 added a core patch that defers unnecessary route rebuilds during batch module installation.

PR #345 β€” Admin Toolbar Performance Patch

The admin_toolbar module's ExtraLinks class called routeExists() for every admin menu link during module installation, potentially triggering a full route rebuild per call. Installing a single module like dblog could hang indefinitely due to cascading route rebuilds. PR #345 added a patch that defers menu link rebuilds via DestructableInterface and batch-preloads routes in routeExists(). Result: drush en dblog -y went from hanging indefinitely to completing in under 10 seconds.

Combined Impact

Metric Drupal 11.1 Drupal 11.3 11.3 + Batch + Patches
Install time 537s 339s 152s (βˆ’72%)
Peak memory 4.38 GB 955 MB 326 MB (βˆ’93%)
SQL queries (menu) 230,127 β€” 113,787 (βˆ’50%)

These optimizations were contributed upstream and benefit every developer and every YMCA association installing YUSAOpenY β€” not just our CI pipeline. Without them, PR Builder would have required a 16 GB RAM server (4Γ— more expensive), and the install sandbox would take 9+ minutes instead of ~2.5 minutes.


Architecture Decisions

SQLite Instead of MySQL

Each PR sandbox needs a Drupal database. We chose SQLite over MySQL for three reasons:

  • No provisioning overhead. Creating a MySQL database requires CREATE DATABASE, CREATE USER, GRANT PRIVILEGES β€” 3 network calls to a shared server. With SQLite, it's just site:install writing a local file.
  • No shared state. MySQL is a shared service; too many simultaneous databases could affect production sites. SQLite is completely isolated.
  • Downloadable databases. Reviewers can download the .sqlite file and inspect it locally β€” a direct link appears in every PR comment.

The tradeoff β€” SQLite doesn't match production MySQL behavior for edge-case queries β€” is acceptable for a review sandbox testing UI, module behavior, and upgrade paths.

Ephemeral Docker Containers with tmpfs

Each build runs in a single throwaway Docker container mounted on tmpfs (RAM disk):

  • RAM is faster than disk. composer install creates thousands of small files. On tmpfs, each file write is a memory allocation β€” no disk seek, no filesystem journal overhead.
  • Clean environment. Each build starts from scratch β€” no leftover files, no stale PHP opcache, no corrupted Composer cache.
  • Resource isolation. Each container gets exactly 1 CPU and 1 GB RAM. A runaway composer install cannot exhaust server memory.

Dedicated PR Builder Node

PR Builder runs on its own node in the sandboxes.y.org cluster, separate from production sites:

sandboxes.y.org cluster:
β”œβ”€β”€ CI node          β€” Jenkins, data server
β”œβ”€β”€ DB node          β€” MySQL 8
β”œβ”€β”€ Hosting node     β€” Caddy (ingress, SSL termination)
└── PR Builder node  β€” dedicated sandbox node (pr-php85 FPM, Caddy fastcgi)

A single Cloudflare wildcard DNS record (*.cibox.tools) and wildcard origin certificate covers all PR sandboxes. Domain scheme: {pr_number}-{type}-pr.cibox.tools (e.g., 123-install-pr.cibox.tools). No per-PR DNS configuration needed.


Pipeline Optimizations

Starting from a naive sequential implementation (~8 min per build), five targeted optimizations brought total build time to ~4:30:

Optimization Before After Savings
Build once, copy everywhere (1Γ— Composer instead of 3Γ—) 270s 90s βˆ’180s
Parallel sandbox setup (install + upgrade + code) 95s sequential 45s parallel βˆ’50s
Native cp -a instead of Ansible copy module 195s (3Γ— 65s) 30s (3Γ— 10s) βˆ’165s
tmpfs build container (RAM disk + single container) ~4 containers + disk I/O ~1 container + RAM βˆ’30s est.
Parallel Docker service discovery (Python ThreadPoolExecutor) 60s (15s Γ— 4) 12s (3s Γ— 4) βˆ’48s

The parallel service discovery script was extracted to jobs/common/scripts/ and applied to all production deployment playbooks β€” saving ~25 seconds on every production site deploy as a side effect.


GitHub Integration

Single Comment, Updated In Place

Instead of posting a new comment on every build, PR Builder creates one comment (identified by an <!-- pr-builder --> HTML marker) and PATCHes it via GitHub API on every rebuild:

### PR Sandbox

| Type    | URL                                      | Status   | Logs                       |
|---------|------------------------------------------|----------|----------------------------|
| Install | https://123-install-pr.cibox.tools       | βœ… ready | build Β· install Β· db       |
| Upgrade | https://123-upgrade-pr.cibox.tools       | βœ… ready | build Β· updb Β· db before Β· db |
| Code    | https://123-code-pr.cibox.tools          | ❌ failed | build                     |

The /retest Command

Contributors can post /retest as a PR comment to trigger a rebuild without pushing a new commit β€” useful for transient failures. Only allowed_authors can trigger /retest. The controller adds a πŸš€ reaction to the comment as immediate confirmation.

The notify Label

GitHub only sends email notifications for new comments, not edits. When the notify label is added to a PR, PR Builder posts separate comments for each sandbox type as they become ready β€” each one triggers a GitHub email notification to subscribers.

Build Logs as HTML Pages

Build logs are saved as HTML pages with ANSI color rendering (via ansi_up.js), preserving Composer and Drush color output (yellow warnings, red errors, green success):

https://logs-pr.cibox.tools/{pr_number}/build.html    β€” composer install + require
https://logs-pr.cibox.tools/{pr_number}/install.html  β€” site:install output
https://logs-pr.cibox.tools/{pr_number}/updb.html     β€” database update output

Results

The PR Builder project ran from March 7–11, 2026: 4 working days, 108 commits, 6 PRs merged.

Ongoing impact:

  • ~55 minutes saved per PR review β€” reviewers click a link instead of setting up local environments
  • ~4:30 build time β€” from push to three live sandbox URLs
  • Zero-configuration testing β€” sandboxes appear automatically within minutes of every push
  • Database upgrade path validation β€” the Upgrade sandbox catches hook_update_N bugs that reach production only on upgrade, invisible on fresh installs
  • Open-source benefit β€” performance improvements in PRs #327, #329, #331, #345 were contributed upstream and benefit the entire YUSAOpenY ecosystem and Drupal core

Why Drupal was chosen

 YUSAOpenY β€” the open-source distribution powering YMCA websites across North America β€” is built on Drupal. The choice was already made: we build and maintain Drupal infrastructure for YMCA associations, and this is the platform the community runs on.

pr builder

Technical Specifications

Drupal version:

Why these modules/theme/distribution were chosen
  • Drupal 11.3 β€” YUSAOpenY distribution
  • Jenkins β€” CI orchestration (cron H/2 * * * * polling)
  • Ansible β€” build and deployment automation
  • Docker + Docker Swarm β€” ephemeral build containers, service orchestration
  • SQLite β€” per-PR database (isolated, downloadable)
  • Caddy β€” ingress proxy + fastcgi for PR sandboxes
  • Cloudflare β€” wildcard DNS + SSL (*.cibox.tools)
  • Drush β€” site:install, updb, cr
  • Composer β€” dependency management
  • GitHub API β€” PR polling, comment management, reaction feedback
  • Python (ThreadPoolExecutor) β€” parallel Docker service discovery
  • Contributed modules patched: admin_toolbar
  • Drupal core issues addressed: #3416522, #3493290, #3492438
pr builder comment