Interview with Chris McMahon

Chris McMahon

Twitter: @chris_mcmahon

Interview

Where are you in your testing journey?

I am a three-time college dropout with some background in Liberal Arts but much more experience as a jazz and pop bass player. I got my first ever forty-hour-a-week real job in 1996 at age thirty-two, doing software customer support. A year later I was testing life critical 911 location mainframe software in the run up to Y2K. I’m fifty-six years old now, I guess I am nearing the end of my testing journey. I still have a few good years left I hope.

The best bug you have ever found?

I am not so much interested in the bugs I did find as much as I am interested in the bugs that I DIDN’T find. I am tempted to tell the story about how a series of cascading failures by management, dev, test, sysadmin, etc. led us to delete millions of 911 location records for a large US state, but I wasn’t directly involved, and it was a long time ago.

Instead I’ll tell you the story of how we corrupted hundreds or maybe thousands of article headers for Wikipedia.

In 2012 I was hired by Wikimedia Foundation (WMF) to create the QA/test practice for Wikipedia. The conversation my first day on the job went something like this:

Me: So you need some software testing done. What do you need tested? Them: You tell us. Me: OK, where is your test environment? Them: We don’t have one.

So my first order of business was to cobble together a test environment for Wikipedia, and start writing tests against it. (The story of how WMF ran Wikipedia for eleven years with no test environment and no testing has to wait for another time.)

So some colleagues and I started creating a test environment for Wikipedia. It was called “beta labs”, and as far as I know it still exists today. Wikipedia is immensely complicated, and as you might imagine, creating a production proxy to test against was not easy, and it took some time. But it improved, gradually, and became more and more useful. But in those early months, we had a LOT of false failures. “Hey, this looks broken” “Oh it’s just beta labs” “Oh, is this broken?” “No, it’s just beta labs”

One day I noticed that a lot of Wiki article headers in the beta labs test environment were garbled, for example the font sizes were radically wrong. “Hey, these fonts look really bad” “It’s just beta labs”

A few days later WMF starts getting reports from users that they are seeing Wikipedia articles in production with garbled article headers and wacky font sizes. HMMM THAT SOUNDS FAMILIAR!

What had happened was that a Wikipedia contributor who was not WMF staff had submitted a PR. Many other Wikipedia contributors, mostly WMF staff, had serious concerns about the quality of the code for the PR, and had refused to merge it. The conversation and updates over this PR had been going on for weeks. The community contributor REALLY wanted their code in Wikipedia, but WMF staff were more and more set against allowing it to be merged, on quality grounds.

What finally happened was that on Thanksgiving Day, on a USA holiday when most of the WMF staff were away, this contributor convinced someone in Europe with commit rights to merge their PR into the mainline Wikipedia code base. I got back from vacation and I saw the garbled article headers in beta labs with my own eyes. I showed it to developers and I showed it to managers. We all said “Oh it’s just beta labs” and we released it to production.

Most problems with Wikipedia can be fixed by reverting code. Not this bug, those garbled headers were permanent problems. Luckily we had a way to identify the garbled article headers. It took one of the senior developers several weeks to fix them all.

That was when I knew I was on the right track in my approach to testing Wikipedia. We took beta labs seriously after that. “

Advice you would give to Testers?

As a tester you should know about how software is made, both by code and by people. My advice:

• Read code
• Write code
• Be gracious and sympathetic and kind when the software fails
• But don’t let bad actors in the company or in the community get away with it”

What does winning mean to you?

I used to want to change the world. And I did! I got standing ovations from big audiences. I played important shows. I was part of the early Open Source movement, worked in the early Agile movement, and the early browser automation efforts. I owe my career to these important waves of practice and technique.

But now I don’t want to change the world so much. I will if I have to, but today “winning” means having meaningful work with good people, for good people, to good ends. Winning means having love and family in my life, and at least a shot at enough money to be comfortable all the way to the end.