PootleConf 2016Posted on by Robbie Cole
Pootle is an open source web application that Skyscanner uses to enable our third party translation suppliers to translate our content. Most people within the business have never seen it, nor even heard its name, but all the application strings in the company have been through our instance of it. Pootle is the power behind the throne of our translation process.
This year the developers of Pootle, Translate House, led by Dwayne Bailey, invited us to their developer conference in London, to share our use cases, our needs and wants, and discuss the future of this vital piece of software. Skyscanner sent engineers Sarah Hale, Eamonn Lawlor and myself (Robbie Cole) to represent.
It began with introductions and backgrounds. We shared what we were broadly interested in talking about and learning over the conference, then launched into deep-dives about how we actually use Pootle.
First, Eoin Nugent from Yelp discussed their translations systems, then Skyscanner internationalisation lead Sarah Hale presented ours.
The Yelp translation process turned out to be pretty similar to Skyscanner’s process. Strings are developed by teams as English source text and then submitted, where they go into Pootle to get translated, then the translations are pushed back to consuming codebases. They still rely on developers synchronising the latest strings manually, while services in Skyscanner will absorb them almost immediately from our RESTful string service, but on the whole, we’re both doing pretty much the same thing. We’re not crazy, and we’re not alone, and that’s tremendously reassuring!
In the afternoon our thoughts turned to open source contributions, to how we could give our tweaks back to the community. The Translate House developers presented their unit and integration testing suites and showed us how to debug Pootle, along with their strategy for coding style and standards enforcement.
Their approach is once again reassuringly aligned with ours. Much as we will not allow code into our own deployments that does not have unit tests, nor will they; when they expressed that they were worried the demand for testing would put people off contributing, we simply shrugged because comprehensive testing is second nature to us.
Getting the test suite up and running in our Windows environments wasn’t quite so smooth, however. After a few reworded command line calls we got a 100% failure rate in 2539 tests, but luckily Eamonn was able to track down a cross-platform file path bug that was at the heart of them all – and contribute the fix back to master!
The second day brought us demonstrations of radical new features and architectural shifts in Pootle. We were first introduced to ‘Pootle File System’, or ‘Pootle FS’, a system that will streamline how Pootle synchronises data in and out of our repositories. This is exactly what we want – built-in functionality that will allow us to remove most of our intermediate bash scripts that connect Pootle to our internal services.
Any integration concerns that Pootle FS wouldn’t cover could then be solved by the next new feature, a plug-in architecture. Rather than hacking onto our own fork of Pootle, thus making up-versioning to get the latest improvements a painful process, developers will be able to build discrete plug-ins that can be installed only in their personal configuration – safely away from danger.
In the afternoon we descended into hackathonning. I had a stab at writing a custom format parser plug-in – the idea being that, instead of having to convert our resource files to an intermediate format for consumption, Pootle would be able to natively understand them and all the wonderful meta-data they contain.
I didn’t quite get all the way in the time available, but did get Pootle at least absorbing the information from our custom file format. We’ll definitely be picking this up again once we’ve migrated to the latest version and can take advantage of all this new good stuff. The theory, however, is already beautiful; this is a future of which we want to be a part.
We also had a discussion about scaling and Translate House’s current work to put Pootle into Docker images. Nothing is quite ready yet, but this area is under active development, so we’re hoping that once these become available our local development and testing capabilities will be much improved – and we’ll be able to launch Pootle instances straight into the cloud.
On the final day we discussed the future of quality checks in Pootle. This is an area of prime interest for us, as we currently have a painful feedback loop – checks are applied after translation is completed, so any failures need to be fed back to the translators for a second pass. What we really want is to use Pootle’s built-in checks that are applied on the spot; when a translator clicks ‘submit’ they will be told immediately about any problems, so they can fix them while their head is still in the zone.
The quality checking framework is up for big changes, as it is currently quite difficult to release and configure checks (they’re housed in the ‘translate toolkit’, a separate project that is still managed by Translate House but isn’t so easy to release). We had a discussion about potential architectures for how checks could be created, configured and managed, trying to find the balance between ease of development, ease of use and future-proofing.
I proposed a super-modular system to separate constraints for running checks from the code behind the checks from their configuration, but my complexity had to be reined in because, yes, Pootle is generally used by actual human beings – often in community situations where people are not as comfortable with technology as we are. There is such a thing as too much flexibility!
With that, we wound down with some blue-sky thinking about what we’d like to see appear next in Pootle. Branded swag was exchanged (with some delightful personalised messages on the swag bags from Translate House!) and proceedings were called to a close to the satisfaction of all.
All in all, it was a fascinating and productive two-and-a-half days. We got to hear about how Pootle works both in similar commercial environments to our own versus how it is used non-profit organisations to manage crowd-sourced community-driven translation.
We got to discuss the future of Pootle, the new features coming soon, and even shape the roadmap a bit by setting out our own use-cases. Best of all, we got to try some of it out, to experience first-hand the new world of improvements that we’ll soon be able to unlock.
It was great to meet some of the Pootle team and other engineers who use Pootle and learn about their processes. Now we’re even more excited to be upgrading our own instance of Pootle, getting to contribute our bits and pieces back to master – and we’re already lining up for PootleConf 2017!