"While there are some excellent job queue systems such as Celery, we didn’t need the whole feature set, nor the complexity of a third-party tool. Leveraging in-house primitives gave us more flexibility in the design and allows us to both develop and operate the Pirlo service with a very small group of SREs."
I don't know any more than this paragraph or two explains, but it sounds like NIH syndrome. If you chose to write your own solution just because a 3rd party one was complicated or expensive, you've underestimated the complexity and expense of developing and supporting new software. Not only do you have software developers developing your business products, but now you have software developers developing the IT tools that support the software developers writing the business products.
"Using the network database and configuration tool developed by our Network Reliability Engineering (NRE) team,"
Another custom tool? Network inventory and config management tools do exist already...
"Rather than having engineers manually running tests using playbooks, Pirlo performed an automated sequential battery of tests that reduced the need for hands-on attention and concurrently increased diagnostic accuracy."
Or you could, like, install Jenkins, write your tests, and do all this without writing your own distributed job queue system.
How does Dropbox manage to LOSE customers' files?
That's a more pressing topic, I think.
A theory based on the article is it seems Pirlo may be written in Python (going off the fact that is leverages SQLAlchemy) - which is interesting given most providers are writing new infrastructure code largely in Go (Spinnaker is another Python exception).
I'm guessing that Python is still heavily used inside Dropbox, but does anyone know if they have published any style guides or tooling to managing Python codebases at their scale?