Mobility 0.2: Now with Plugins

It’s been a little while since last I posted about Mobility, the pluggable translation framework for Ruby that I’ve been working on recently. I thought I’d take this opportunity with the release of the second version of the gem to highlight some of the important changes and new features, and to recap how far things have come.

Mobility and the Module Builder Pattern

In the time since I posted Translating with Mobility, one of the techniques I have used heavily in building the gem – the Module Builder Pattern, as I’ve termed it – has become a hot topic of its own. (I’ll actually be talking about module builders at the coming RubyKaigi in Hiroshima, check it out if you’ll be attending.)

In a nutshell, the Module Builder Pattern is a simple way to create customizable modules to mix into classes, one which falls naturally out of Ruby’s object model. It amounts to simply subclassing the Module class, defining module methods in an initializer or included hook, and including instances of the subclass in other classes. See the slide below from my recent talk at the Tokyo Rubyist Meetup, where I explain the basic elements of a module builder and how I use it in a core Mobility class, Mobility::Attributes.

From its first release, Mobility has used this pattern all over the place; I could never have achieved the flexibility the gem needed without it. However, in the latest release, I’ve pushed this yet further in order to achieve something just as important: extensibility.

Now with Plugins

Let’s recall that in Mobility, you define a translated attribute on a model by calling translates with one or more attribute names, and a hash to configure which options are applied: dirty-tracking, fallbacks, locale accessors, and so on. Under the hood, Mobility creates an instance of the Mobility::Attributes module builder and includes it in the class. I’ll use this form directly here to emphasize what is actually going on:

class Post
  extend Mobility
  include Mobility::Attributes.new("title", "content", locale_accessors: [:en, :ja], fallbacks: true)
end

Here, we are building a module instance of Mobility::Attributes for the attributes title and content, with locale accessors for English and Japanese and fallbacks enabled. (The format of arguments to this initializer have changed slightly since version 0.1, see this pull request for details).

What happens when we include the instance of this class into Post? Well, here’s the first change with version 0.2: I’ve made the magic here a whole lot clearer, and more extensible as a result (see PRs #50 and #62).

Previously there was a lot of hard-coded references to options directly in Attributes, but the code is now entirely free of such references. In fact, there are just a few lines in the builder which deal with options:

Mobility.plugins.each do |name|
  plugin = get_plugin_class(name)
  plugin.apply(self, options[name])
end

Here, Mobility.plugins is an alias for Mobility.config.plugins, a new configuration setting which defaults to this array (order of elements is important here¹):

[:cache, :dirty, :fallbacks, :presence, :default, :fallthrough_accessors, :locale_accessors]

The private get_plugin_class method fetches constants under Mobility::Plugins matching CamelCase-ed values of the plugin names from this array, then applies them with the attributes instance and option values as its arguments.

So locale_accessors: [:en, :ja] will fetch Mobility::Plugins::LocaleAccesors and call it like this:

Mobility::Plugins::LocaleAccessors.apply(self, [:en, :ja])

…where self is the instance of Mobility::Attributes being included into the model class, Post.

Notice that since we have no hard-coded references to option keys in this module now, any option key included in Mobility.plugins will apply a plugin with that name (as long as that plugin exists), with the attributes instance and option value passed into the plugin’s apply class method.

This makes is very easy to design new extensions: all they need is to have a class method, apply, which takes this Attributes instance (which itself has references to the backend class and attribute names), and an option value to configure how the plugin should be applied. (Incidentally, I borrow the term “plugins” here, and the method name apply, from Sequel, where plugins are defined in a somewhat similar way.)

Below is another slide from my recent talk, illustrating how these options trigger inclusion of yet more module builder instances into the backend class and into the Attributes instance itself.

Along with this introduction of plugins and the Mobility::Plugins namespace, I’ve also reorganized the backend namespace such that Mobility::Backends (plural) is now exclusively used for backend classes.

The result is that it becomes very straightforward to build custom backends and plugins: define a class in the appropriate namespace with the required structure, and call translates (or include an instance of Mobility::Attributes) in the model class, with any backend or option keys, and that’s it! You ready to go.

Default Options

Another minor annoyance some users of Mobility may have encountered in 0.1 is that while it was possible to set a default backend to use across all models (with the default_backend configuration option), there was no way to define a default set of options (to configure plugins and the backend). Along with the changes to enable plugins above, I also added this as a configuration setting, with a value that defaults to:

{ cache: true, dirty: false, fallbacks: nil, presence: true, default: nil }

These default option values were previously hard-coded, but are now explicitly set and isolated in the configuration instance, which makes them much easier to reason about and change.

Want to enable fallbacks by default on all your models? Just override the default for that key:

Mobility.default_options[:fallbacks] = true

You can also set backend options here, so for example if you want to set the default type for the KeyValue backend to be “string” instead of “text”, you could do this:

Mobility.default_options[:type] = :string

Note that how these option values are interpreted by actual plugins and backends depends on the plugin or backend, and there is some unavoidable variation there. But the values passed in to apply (plugin) and new (backend) are entirely determined from this hash.

Enumerable Backends

Backends as implemented prior to version 0.2 could only do two things: read and write values for a given locale. If you wanted to know what locales were available for a given attribute, you would be stuck.

This was raised as an issue, and I decided that rather than just add a method returning locales, I’d go a bit further and try to solve a more general problem: iterating through translations. I implemented this in PR #71 by adding a new each_locale method to each existing backend returning successive locales to a block.

From these backend-specific each_locale methods, an each method is defined in Mobility::Backend yielding instances of a Translation struct wrapping the locale and backend. We then include Ruby’s powerful Enumerable module, giving us with it a whole slew of traversal and search methods. A method locales is also added to Mobility::Backend which returns the available locales for the attribute (the original request).

With this change, you can now do things like select translations that match a condition:

post = Post.create(title_en: "English foo", title_de: "German foo", title_ja: "Something else")
post.title_backend.select { |t| t.read =~ /foo/ }.map &:locale
#=> [:en, :de]

Although this is a somewhat contrived example, it’s not hard to see that having the full arsenal of Enumerable methods at your disposal could be quite useful in a variety of contexts.

Autoload → Require

Mobility 0.1.x uses autoload to load nearly all its constants, but as Rubyists no doubt know, autoload has its issues, and may even be deprecated in a future release of Ruby. So I decided to bite the bullet and convert all autoload calls to require (here’s the PR). For this I followed the pattern (again) of Sequel, which only requires plugins when they are called with the plugin method.

So Mobility now does a similar thing for backends and plugins. Here is the code loading a backend when it is triggered from a call to translates:

def get_backend_class(backend)
  return backend if Module === backend
  require "mobility/backends/#{backend}"
  get_class_from_key(Mobility::Backends, backend)
end

Rather than simply finding the constant, we first require the backend at "mobility/backends/#{backend}", then once we have loaded the file we make a const_get call to actually fetch the constant (which previously would have triggered autoload to find the backend file in the same location).

Admittedly, this now requires the plugin/backend creator to put the backend/plugin in the requisite file location or else Mobility will not find it. But that seems to me a quite reasonable thing to do anyway.

It also means that you can’t just require "mobility", and then expect Mobility::Backends::KeyValue (or any backend class) to be loaded, because it will not be required until you actually use it. You need to either call translates, or require the class explicitly, to load the backend.

Mobility::Util

Continuing with the autoload clean-up, I also decided to fix something else that had been bothering me for a while: mobility/core_ext. When I originally decided to make Mobility support Sequel in addition to ActiveRecord, I found it hard to entirely give up ActiveSupport since (like many Rubyists) I had become so accustomed to using methods like present?, camelize, singularize, etc.

So by default, if ActiveSupport could not be loaded, I included a bare minimum implementation of just a few of these methods, filed under mobility/core_ext. In addition, for Sequel backends, I also just assumed that the Inflections module was loaded, otherwise the backends would not work (I required it in Mobility’s Sequel specs).

In this PR, I replaced mobility/core_ext, which (like ActiveSupport) monkeypatched core classes like Object and String, with a module Mobility::Util which contains all the methods needed. Util can be either included in a class, or its methods can be called as class methods: e.g. Util.present?(string) will call string.present? if the object responds to present?, otherwise it will do the work itself to get the result.

Now Sequel does not need to include the Inflections module to use Mobility, and the core parts of Mobility (e.g. Mobility::Attributes) do not require any special monkeypatching to work. This is important for anyone wanting to use Mobility for an application where they do not want to patch core Ruby classes (a good thing).

Super as Option

In some cases, Mobility overrides a method to redefine it as a translation accessor. This is the case, for example, with backends that store translations on a single column: the PostgreSQL (Jsonb + Hstore) backends, as well as the Serialized backend (which stores translations serialized as a json or yaml hash).

With these backends, the original attribute method, say title, returns a hash containing all translations as key/value pairs (where locales are the keys). Mobility overrides this column accessor to return value of the hash at the current locale, using (for AR models) read_attribute to avoid infinite recursion (Sequel is similar).

A few people had asked about how they could access the original (hash) column, so in this PR I added a new super option which bypasses the Mobility accessors (reader and writer) and continues up to the original method, in this case the column method.

So something like this:

post.title
#=> "Translating with Mobility"
post.title(super: true)
#=> { "en" => "Translating with Mobility", "ja" => "Traduction avec Mobilité" }

The writer can also be “super-ed”, although this is somewhat trickier to do:

post.send(:title=, {}, super: true)
#=> {}
post.title
nil

Other

• The default name for associaitons in the KeyValue and Table backends has been simplified. KeyValue associations named mobility_string_translations and mobility_text_translations are now simply string_translations and text_translations, respectively. For the Table backend, mobility_model_translations has been simplified to simply translations (like Globalize).
• In PR #58, I refactored a somewhat ugly part of the code dealing with caching. I’m still not completely happy with how this is done, but it’s at least an improvement over the earlier technique.
• I now have some minimal object allocation specs, added in PR #57. I’d like to extend this to benchmark testing as well, but not quite sure how to proceed yet on that one (any suggestions welcome!)
• The gem is now signed. You of course do not need to check this signature, but if you want to, there are instructions in the README on how to do this.
• extend Mobility is now the preferred way to use Mobility in models (see this issue).
• I’ve bumped Ruby support to minimum 2.2.7. This is pretty standard.

Finally, a few words

Mobility is a labour of love, and I think it’s worthwhile to mention that here. I’ve spent countless hours writing and re-writing Mobility’s code, and many more thinking and rethinking how I can refactor and improve that code.

I have both selfish and unselfish motivations for doing this: on the one hand, I want to show off the best that I can do with a clean slate and a clear problem (translation) that I happen to know reasonably well. On the other, I genuinely want to help people who are trying to share content across language barriers, a problem which has been an interest of mine for a long while.

So it’s been enormously gratifying for me to see the positive response Mobility has received so far in the form of encouraging comments and reactions, a growing list of stargazers, and as actual contributions to the gem code. I’ve had very thoughtful input on edge cases and subtle points that I would never have considered, as well as some great feedback at recent talks in Montreal and Tokyo (see slides from the latter of those talks below).

There are many horror stories about open source software maintainers getting exhausted from the demands that come with success. Mobility hasn’t reached anywhere near that level of success yet, but so far at least I don’t find myself exhausted at all. On the contrary, it’s been an extremely rewarding learning experience!

If you’re using Mobility for a project or application, please let me know. I’m eager to here about the kind of challenges people are tackling and how Mobility might be better able to help solve them.

Notes

¹ The order of plugins in Mobility.plugins is important: many of the plugin classes include modules into the backend overriding read and write, inclusion order determining the order of method composition. In the list, for example, cache comes before default (and included in that order) because we want to cache actual backend values, not default values.