Facebook and I Shared a Similar Approach to E2E Test Automation and Continuous Testing
Product Strategy: “Release Early; Release Often”; Watir -> Selenium WebDriver; Developed own IDE and Continuous Testing Server; Using Mac Minis in the on-premise parallel testing lab.
My professional experience never intersected with Facebook, and I’ve never met anyone who worked there (though I do have a Facebook account, with zero posts so far). Until June 20, 2015, I was completely unaware of how Facebook approached E2E test automation and Continuous Testing. That changed when I came across an excellent presentation on YouTube from the F8 Developer Conference, where Katie Coons shared insights on how Facebook implements E2E test automation and CI.
Please note: the claim of “Similar” is based on Facebook’s published practices in 2015. I don’t know how Facebook does E2E test automation or Continuous Testing now, frankly, I am not that interested. Why? Because my open and independently-developed approach (including tools) has been highly effective for me over the past 18 years, barely changed.
By that time, I had already been working on E2E test automation and Continuous Testing for 10 years and had also developed a few side-hustle apps. To my surprise, I realized Facebook’s approach shared many similarities with my own practices.
Table of contents:
· Product Strategy: “Release Early; Release Often”
· Started with Watir
· Upgrade to Selenium WebDriver
· Developed IDE-like tool to support better Scripting E2E tests
· Developed a Continuous Testing Server to support executing automated E2E tests
· Used Mac Minis in the Test Lab
· You can use TestWise and BuildWise free
Product Strategy: “Release Early; Release Often”
It all began with Product Strategy. As a software solopreneur, the primary reason I embraced end-to-end (UI) test automation was to enable frequent app releases to production—daily or even hourly. While many software professionals understand the benefits of frequent releases, my perspective goes a step further: releasing often hinges on reliable automated E2E (UI) regression testing. This understanding is why I dedicated significant time and effort to mastering E2E test automation, since 2005. (check out my article, My Test Automation Journey).
Just a few days ago (January 2025), I came across the video below (shared in this LinkedIn post) where Mark Zuckerberg discussed his product strategy.
It’s quite similar to the approach I had when I founded my company, AgileWay, back in 2006. Some readers might think, “Zhimin, are you boasting by comparing yourself to Mark Zuckerberg?” That’s not my intention at all. Of course, I can’t compare myself to Zuckerberg—I’m just an unknown solopreneur, while he’s one of the richest and most well-known people on the planet.
However, despite the vast differences in social status and wealth, two people can share similar thinking in a certain area, right? Long-time readers of my blog would know that I’ve backed up my thoughts with evidence over the years.
I will write a separate article decoding this product strategy. In this article, just focus on the similar technical practices Facebook and I both took.
Started with Watir
Watir stands for "Web Application Testing in Ruby." It automates web controls in Internet Explorer, the dominant browser of its time, through OLE (Object Linking and Embedding) using Ruby scripts.
Facebook:
“For browser-based testing of our Web code, we use the Watir framework. We have Watir tests covering a range of the site’s functionality, particularly focused on privacy — there are tons of “user X posts item Y and it should/shouldn’t be visible to user Z” tests at the browser level.” — Steven Grimm, the testing engineering lead at Facebook (2005–2012), responded on Quora.
Special note: the Facebook app was developed in PHP, and it adopted Ruby as the scripting language for E2E test automation.
Me:
My first exposure (in 2005) to E2E test automation was JWebUnit, it was natural to me as a senior Java developer. Then, I discovered Watir, and soon I liked it a lot, especially the Ruby language. I even wrote an ebook on Watir.
Check out my article, “Why Ruby is the Best Scripting Language for End-to-End Test Automation?”
Upgrade to Selenium WebDriver
Facebook:
Video clip from “Continuous Integration at Facebook”
Me:
I appreciate the initiative of Selenium v1, but I wasn't fond of its approach of using JavaScript to drive web elements. When Selenium 2 (Selenium WebDriver) was released in 2011, I quickly adopted it and integrated support for it into my testing IDE.
Check out my article: “Why do I prefer Selenium (Ruby) over Watir?”
Developed IDE-like tool to support better Scripting E2E tests
Facebook: Nuclide
I know little about Nuclide (based on Atom), besides this presentation.
“We built our own IDE, We called it Nuclide” - Christian Legnitto, Engineering Manager, Release Engineering, Facebook (April 2015, at F8 conference)
If you watch the video, you will find out Microsoft did a clone of it: Visual Studio Code (first release, 2015). You get the idea of what it is.
Me: TestWise
Also recognizing the need for a good testing tool, different from Facebook’s Nuclide, I created a dedicated functional testing IDE: TestWise.
The first build of TestWise, initially named iTest2, was completed on April 11, 2006, following about three years of development in my spare time. The first public release occurred on January 5, 2019. Today, I am still using use TestWise daily (if doing work), for over 18 years. (For more, check out the article, My Innovative Solution to Test Automation: TestWise, the Next-Gen Functional Testing IDE)
Developed a Continuous Testing Server to support executing automated E2E tests
Facebook chose not to use existing commercial or open-source CI tools like Bamboo, TeamCity, or Jenkins. Why? While these CI servers are capable of running automated end-to-end (E2E) tests, they aren't well-suited for the task at scale.
Recognizing this limitation—just as I did—Facebook developed its own Continuous Testing Server called Sandcastle. However, they kept Sandcastle proprietary, with very little information available about it outside of a conference video (in 2015) I mentioned earlier. It seems Facebook treated Sandcastle as a strategic advantage, keeping it as their secret weapon.
In 2008, I recognized the limitations of CI servers for executing E2E tests. To address this, I began designing & implementing continuous testing (CT) features (such as auto-retry, dynamical ordering, manual-retry, and delay completion. To my knowledge by Google searching, no one was doing that, that’s why I came up with my own solutions), added to the open-source CruiseControl server (the granddaddy of all CI), utilizing custom-built agents. This eventually led me to develop my own fully free and open-source CT server, BuildWise, which won the Ruby International Award in 2018. (For more, check out My Continuous Testing Journey)
Used Mac Minis in the Test Lab
Facebook:
Below were taken from the “Continuous Integration at Facebook” presentation.
Wow!
Mine:
As a solopreneur, I lack resources. My test lab has only changed slightly since 2012—it's still mostly just a few Mac Minis, reminiscent of Facebook's setup back then. While I’ve added some new Mac Minis (the M1 Mac Minis and the latest M4 Mac Mini), I still have two Mac Mini 2012 in my testing lab. (Yes, they have been running E2E tests for me, for about 12+ years).
You can use TestWise and BuildWise free
BuildWise Server is 100% free and open-source. Both TestWise IDE and BuildWise Agent have a free mode, fully featured with just minor constraints:
TestWise free mode: up to 25 test executions per launch.
the launch time only takes seconds.BuildWise Agent has a free running time of 40 minutes
a bit like Zoom.
Some software professionals might be impressed by the fact that I created complex tools like the TestWise IDE and Continuous Testing server in my spare time. Honestly, I didn’t think much of it—end-to-end test automation was (and still is) fun for me, and it made me a much better programmer at work. I simply wanted to keep improving in this area. Above all, there was also NO commercial pressure, and I just kept pushing forward year after year.
That said, this could be inspiring for potential solopreneurs. So, take action!
Related reading: