Recently I need to call Google Content API to access Google Merchant Center by using Ruby. I googled on the internet but didn’t find useful links, and also the google-api-client gem doesn’t contain a lot of examples. Thanks David Tzau for his excellent blog, however he uses Python, and in this blog I will show how to call Google Content API by using Ruby.

Create the Client ID for a Service Account

In my application I will use Service Account, which means that the application doesn’t redirect to the Google login page and asks the user to login, instead the application calls the Content API on behalf of the Service Account. This step is the same as David’s blog, so I’ll copy & paste here shamelessly.

So let’s create a Service Account in Google Developer Console. Go to the Google Developer Console and navigate to the “Credentials” menu under “API’s and Auth”. From there click on “Create new Client ID”:

Select the “Service Account” option:

As soon as you create the Service Account, the system will generate a Client ID, email address, and a public/private key pair. The private key is immediately downloaded to your browser. Save this private key safely somewhere. There is a password that comes with this private key that you will need later. The password is “notasecret” and is the same for every private key that is generated.

You will notice a client ID and email address are also created for the Service Account you just created. Make note of the email address as you will need this later:

Granting Permissions inside Google Merchant Center for the Client ID

This next step is very important and is not called out very clearly in Google’s documentation and is very easy to miss. Just because you can digitally sign JWT’s and Google’s authentication server can verify your application is calling on behalf of the Service Account, it doesn’t mean your application can access just any account’s data behind the API whether that be Google Analytics, Google Merchant Center, Adwords, etc. In this example we are going have our program call the Content API for Shopping to retrieve a product from the corresponding Google Merchant Center account.

Login to the corresponding Google Merchant Center Account and navigate to the “Users” menu under “Settings”. Enter the Service Account email address here and grant it “Standard” access.

Add google-api-client Gem

To calling Google Content API, we need to add google-api-client Gem into our Gemfile.

1

gem "google-api-client"

And then run bundle install

Get Google::APIClient

To calling Google Content API, we need an Google::APIClient instance.

1
2
3
4
5
6
7
8
9
10

key = Google::APIClient::PKCS12.load_key("#{Rails.root}/config/content_api.p12",
      'notasecret')
asserter = Google::APIClient::JWTAsserter.new(ENV['GOOGLE_CONTENT_API_CLIENT_ID'],
  'https://www.googleapis.com/auth/content', key)

client = Google::APIClient.new(
  :application_name => 'Your App',
  :application_version => '1.0.0')
client.authorization = asserter.authorize
content = client.discovered_api('content', 'v2')

Remember the private key file we downloaded when we create the Service Account? I saved the file in #{Rails.root}/config with file name content_api.p12 (If your repository is an open source repository, you should not put this file in your repository as it’s very sensitive).

On line 1, it load the key from the content_api.p12, and on line 3 it creates an asserter instance. The ENV[‘GOOGLE_CONTENT_API_CLIENT_ID’] is the Client ID as indicated in the following snapshot. Also you should not put this information in your repository if your repository is public. For my project I put it in a yml file and when the application loads, it will set the key/value in the yml file as environment variables.

On line 6 it creates a Google::APIClient instance, and on line 9 it sets the authorization.

On line 10, it calls client.discovered_api to get the content api, we will use this object to dictate which api we will call when we call the api, as will show in following sections.

Calling Content API

Request

To call Google Content API, normally we would call the Google::APIClient#execute method. This method accepts a Hash parameter, and the Hash normally contains the following options,

:api_method

The :api_method indicates which API we would like to call. Normally we set this option by calling the content object we got above. To know which API to call, take a look at the Google Content API reference.

For example, to call the Accounts: insert API, as the figure below, we should set :api_method to content.accounts.insert

:parameters

The parameters should be a hash, it indicates the path parameters in the request URL, for example, the Accounts: insert API expects a merchantId param, so to call this API, it should set the parameters to { :merchantId => merchant_id }. Notice that the parameter key is camel case, which doesn’t conform to Ruby convention.

:body_object

If the API contains a body, we should put the body in the :body_object option. The :body_object should be a Hash, what it contains depends on the specific API.

For example, to create an Account with name ‘my_account’, we should set the :body_object as a hash like this,

:body_object => { :name => ‘my_account’ }

Response

When call the execute method, it returns us a response, we should call response.data? to check if the response contains any data. And if the API is failed, we can check response.data[‘error’]. If the API calling is successful, we can check the documentation to see what the response data contains.For example, for Accounts Insert, if successful it will returns a Accounts resource, so we can call response.data[‘id’] to get the ID of the created Account.

Examples

Now we give some examples as following. In the examples, the client is a Google::APIClient instance. Normally you should wrap this client instance in a class, and make the methods as instance methods of the class.

Create Merchant Center sub-Account

To create a sub-Account, and also give an email as the admin, we will call the following API,

1
2
3
4
5
6

def create_merchant_center_account(parent_merchant_id, account_name, admin_email)
  client.execute(:api_method => content.accounts.insert, :parameters => { :merchantId => parent_merchant_id },
    :body_object => { :name => account_name,
      :users => [ { :emailAddress => admin_email, :admin => true }]
    })
end

This method will create a new sub-Account under the parent_merchant_id, and set the admin_email as the admin of the created sub-Account.

Delete sub-Account

To delete a sub-Account, call the method like following,

1
2
3
4

def delete_merchant_center_account(parent_merchant_id, account_id)
  client.execute(:api_method => content.accounts.delete, :parameters => { :merchantId => parent_merchant_id,
    accountId: account_id })
end

It will delete the sub-Account whose ID is account_id from the parent_merchant_id.

Create a feed

To create a feed, call the API like following,

1
2
3
4
5
6
7
8

def create_merchant_center_feed(merchant_id, filename, url)
  client.execute(:api_method => content.datafeeds.insert, :parameters => { :merchantId =>   merchant_id },
    :body_object => { :contentLanguage => 'en', :contentType => 'products', :fileName => filename,
      :attributeLanguage => 'en',
      :name => filename, :targetCountry => 'AU',
      :fetchSchedule => { :fetchUrl => url, :hour => 8 }
    })
end

The above method will create a feed which targets Australia and the language is english. And we set the fetch schedule as 8am of the day.

In this part we will continue our exploration of how Rails finds your templates. If you need a refresh, here is Part1, Part 2 and Part 3.

In last part we saw that ActionView::LookupContext#find_template is just a delegation to ActionView::PathSet#find. So let’s check what ActionView::PathSet#find is doing.

ActionView::PathSet

From the class name of PathSet we can get an idea that this class is to manage a set of pathes, where the templates are stored. Let’s have a look at the definition of this class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

module ActionView #:nodoc:
  # = Action View PathSet
  #
  # This class is used to store and access paths in Action View. A number of
  # operations are defined so that you can search among the paths in this
  # set and also perform operations on other +PathSet+ objects.
  #
  # A +LookupContext+ will use a +PathSet+ to store the paths in its context.
  class PathSet #:nodoc:
    include Enumerable

    attr_reader :paths

    delegate :[], :include?, :pop, :size, :each, to: :paths

    def initialize(paths = [])
      @paths = typecast paths
    end

    def find(*args)
      find_all(*args).first || raise(MissingTemplate.new(self, *args))
    end

    def find_all(path, prefixes = [], *args)
      prefixes = [prefixes] if String === prefixes
      prefixes.each do |prefix|
        paths.each do |resolver|
          templates = resolver.find_all(path, prefix, *args)
          return templates unless templates.empty?
        end
      end
      []
    end

    def exists?(path, prefixes, *args)
      find_all(path, prefixes, *args).any?
    end

    private

    def typecast(paths)
      paths.map do |path|
        case path
        when Pathname, String
          OptimizedFileSystemResolver.new path.to_s
        else
          path
        end
      end
    end
  end
end

• We can see that the initialize method accepts a paths param, which is normally an array of strings. And in initialize method, it will call the typecast the paths and then store the typecasted value in @paths instance variable
• for each element in the paths, the typecast method will convert it to an OptimizedFileSystemResolver instance.
• The find_all method will interate each resolver instance in paths and return the first value which is not empty
• The find method just delegates to find_all method, if it can’t find a template, it will raise MissingTemplate exception.

In LookupContext class, we can see how the ActionView::PathSet instance is created.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

module ActionView
  class LookupContext

    # Helpers related to template lookup using the lookup context information.
    module ViewPaths
      attr_reader :view_paths

      # Whenever setting view paths, makes a copy so that we can manipulate them in
      # instance objects as we wish.
      def view_paths=(paths)
        @view_paths = ActionView::PathSet.new(Array(paths))
      end
    end

    include ViewPaths
  end
end

The view_paths= method is defind in module ActionView::LookupContext::ViewPaths and ActionView::LookupContext includes this module. We can see that the when calling view_paths= method, an instance of ActionView::PathSet will be created and set to the @view_paths instance variable of ActionView::LookupContext.

So how this view_paths= method is called? It’s in ActionView::LookupContext#initialize method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

module ActionView
  class LookupContext

    def initialize(view_paths, details = {}, prefixes = [])
      @details, @details_key = {}, nil
      @skip_default_locale = false
      @cache = true
      @prefixes = prefixes
      @rendered_format = nil

      self.view_paths = view_paths
      initialize_details(details)
    end

  end
end

So the view_paths is passed to ActionView::LookupContext instances in initialize. And in ActionView::ViewPaths it initializes ActionView::LookupContext.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

module ActionView
  module ViewPaths
    extend ActiveSupport::Concern

    included do
      class_attribute :_view_paths
      self._view_paths = ActionView::PathSet.new
      self._view_paths.freeze
    end

    # LookupContext is the object responsible to hold all information required to lookup
    # templates, i.e. view paths and details. Check ActionView::LookupContext for more
    # information.
    def lookup_context
      @_lookup_context ||=
        ActionView::LookupContext.new(self.class._view_paths, details_for_lookup, _prefixes)
    end

    def append_view_path(path)
      lookup_context.view_paths.push(*path)
    end

    def prepend_view_path(path)
      lookup_context.view_paths.unshift(*path)
    end

    module ClassMethods
      # Append a path to the list of view paths for this controller.
      #
      # ==== Parameters
      # * <tt>path</tt> - If a String is provided, it gets converted into
      #   the default view path. You may also provide a custom view path
      #   (see ActionView::PathSet for more information)
      def append_view_path(path)
        self._view_paths = view_paths + Array(path)
      end

      # Prepend a path to the list of view paths for this controller.
      #
      # ==== Parameters
      # * <tt>path</tt> - If a String is provided, it gets converted into
      #   the default view path. You may also provide a custom view path
      #   (see ActionView::PathSet for more information)
      def prepend_view_path(path)
        self._view_paths = ActionView::PathSet.new(Array(path) + view_paths)
      end

      # A list of all of the default view paths for this controller.
      def view_paths
        _view_paths
      end

  end
end

This ActionView::ViewPaths will be included in a controller class. We can see that by default the view_paths is an empty array. But it provides append_view_path and prepend_view_path to add a view path at the end or the front of view paths. Notice that the prepend_view_path and append_view_path are defined both as class methods and instance methods.

So how the view path is initialized? Actually it’s in Rails::Engine class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

module Rails

  class Engine

    initializer :add_view_paths do
      views = paths["app/views"].existent
      unless views.empty?
        ActiveSupport.on_load(:action_controller){ prepend_view_path(views) if respond_to?(:prepend_view_path) }
        ActiveSupport.on_load(:action_mailer){ prepend_view_path(views) }
      end
    end

  end

end

a initializer defines a code block which will be called during application startup. So when load :action_controller it will call prepend_view_path and the view_paths contains only one element, which is the app/views folder in your rails application.

OptimizedFileSystemResolver

Now understand how ActionView::PathSet is initialized, and we also know that in find method it actually delegates to the resolver. So let’s see how the resolver implements the method.

The Resolver is to resolve a template when passing the details. The class relationship is as the above diagram,

We can see that by default the resolver used in ActionView::PathSet, which is ActionView::OptimizedFileSystemResolver, extends from ActionView::FileSystemResolver, which in terms extends from ActionView::PathResolver, which extends from ActionView::Resolver, which is the base class for all resolvers.

The base class ActionView::Resolver provides some common functionalities such as caching for all sub-class resolvers. However here I will focus on ActionView::PathResolver, which implements the logic how to find a template in a path.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

module ActionView

  class PathResolver < Resolver

    EXTENSIONS = { :locale => ".", :formats => ".", :variants => "+", :handlers => "." }
    DEFAULT_PATTERN = ":prefix/:action{.:locale,}{.:formats,}{+:variants,}{.:handlers,}"

    def initialize(pattern=nil)
      @pattern = pattern || DEFAULT_PATTERN
      super()
    end

    private

    def find_templates(name, prefix, partial, details)
      path = Path.build(name, prefix, partial)
      query(path, details, details[:formats])
    end

    def query(path, details, formats)
      query = build_query(path, details)

      template_paths = find_template_paths query

      template_paths.map { |template|
        handler, format, variant = extract_handler_and_format_and_variant(template, formats)
        contents = File.binread(template)

        Template.new(contents, File.expand_path(template), handler,
          :virtual_path => path.virtual,
          :format       => format,
          :variant      => variant,
          :updated_at   => mtime(template)
        )
      }
    end

    def find_template_paths(query)
      Dir[query].reject { |filename|
        File.directory?(filename) ||
          # deals with case-insensitive file systems.
          !File.fnmatch(query, filename, File::FNM_EXTGLOB)
      }
    end

    # Helper for building query glob string based on resolver's pattern.
    def build_query(path, details)
      query = @pattern.dup

      prefix = path.prefix.empty? ? "" : "#{escape_entry(path.prefix)}\\1"
      query.gsub!(/\:prefix(\/)?/, prefix)

      partial = escape_entry(path.partial? ? "_#{path.name}" : path.name)
      query.gsub!(/\:action/, partial)

      details.each do |ext, variants|
        query.gsub!(/\:#{ext}/, "{#{variants.compact.uniq.join(',')}}")
      end

      File.expand_path(query, @path)
    end

    def escape_entry(entry)
      entry.gsub(/[*?{}\[\]]/, '\\\\\\&')
    end
  end

end

The most important methods are build_query and find_template_paths.

• The PathResolver will use a pattern to search templates in path, and by default the pattern is DEFAULT_PATTERN
• build_query: This method is to build a query from the path and details. Remember in last part, ActionView::AbstractRenderer#extract_details will extract details from options. For example, for an index action in Articles controller, the query will be

articles/index{.en,}{.html,}{+:variants,}{.haml,}

For this query, in find_template_paths it will call Dir[query], which is actually an alias to glob which is to expand the query. For example, for the above query, {p, q} matchs either p, or q. So for example for {.en,} it could either match .en or an empty string. So if we define a template articles/index.html.haml, it will match the query. For more information, consult Ruby doc.

So for each matched template, in query method it will create a Template to wrap the template file.

Summary

So in summary here we can see that the flow for Rails to find a template is as following,

1. Rails initializes a set of Paths for resolve templates, by default it contains only one path app/views
2. When an action of a controller is accessed, the render will be called either explicitly or implicitly.
3. Rails will extract the prefixes, action and some details from the options passed to render,
4. Rails find the template by search in the path set using a query composed from prefixes, action and details.

Before I worked in a large company which has its own DBAs. Yes I know SQL,I know Stored Procedures and I think I know something about indexes, but for database performance tuning, we always delegate the task to the DBA. And currently I work in a small company which doesn’t have a specific DBA role, so developers have to be DBA (Full Stack right?). So for learning something about database performance tuning, I bought this book. Oh I really love it and I learned so many things in one week!

This book talks about indexes and only one type of indexes: B-Tree. It firstly starts with the basics of index and the WHERE clause. And then proceed to join, clustering, sorting and grouping, and also partial results and modifying data. After reading this book, you should have a firm understanding how indexes work and I’m sure that this is needed for every Rails developer. As I totally agree the author’s claim, that SQL performance tuning is not just a DBA task but a developer’s task, as the performance is related to business logic and only developers understand that. And also we should think performance from design, and it’s not an afterthought.

PS: There are simpler solutions for Devise multitenancy authentication, such as RailsCasts and this blog, but customizing Devise gave me an excuse to explore the source code of Warden and Devise though :).

Last year I read the book Multitenancy with Rails written by Ryan Bigg. In the book when he implements the authentication for the multi-tenancy, he uses Warden instead of Devise, cause using Devise will need a lot of customization. And Recently I need to implement this multi-tenancy authentication for a Rails application, and I want to use Devise, so I checked the source code of Warden and Devise and found a way to customize Devise to do it.

In my application, I have two models, Merchant and User, one user can be the owner of one or more merchant, so they are a has_and_belongs_to_many relationship.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

class Merchant < ActiveRecord::Base

  EXCLUDED_SUBDOMAINS = %w(admin www administrator admins owner)

  validates_exclusion_of :subdomain, in: EXCLUDED_SUBDOMAINS,
    message: "is not allowed. Please choose another subdomain"

  validates_format_of :subdomain, with: /\A[\w\-]+\Z/i, allow_blank: true,
    message: "is not allowed. Please choose another subdomain."

  validates :subdomain, presence: true, uniqueness: true

  mount_uploader :avatar, AvatarUploader

  before_validation do
    self.subdomain = subdomain.to_s.downcase
  end

  has_and_belongs_to_many :owners, class_name: 'User', join_table: 'merchants_owners'

  def owner?(user)
    return false unless user.present?
    !!owners.find_by(id: user.id)
  end

end

class User < ActiveRecord::Base

end

Each merchant can access its web interface through a subdomain URL. In model Merchant there is a unique attribute subdomain, this will be used for subdomain. For example, one merchant’s subdomain is skywatch, then the URL for this merchant is http://skywatch.xhop.pe . And for the authentication, a user can login this URL only if he is this merchant’s owner.

And in Merchant class we defined an instance method owner? of check if a user is owner of a merchant.

Subdomain Routes Configuration

For all controllers related to a merchant, I put the controllers under a module Merchant.

1
2
3
4
5
6
7
8
9

constraints(Constraints::SubdomainRequired) do
  root 'merchant/dashboard#index', as: :merchant_root

  namespace :merchant do

    resources 'products'
    # and other resources
  end
end

We set a constraint Constraints::SubdomainRequired to match merchant’s routes.

1
2
3
4
5
6
7

module Constraints
  class SubdomainRequired
    def self.matches?(request)
      request.subdomain.present? && request.subdomain != "www"
    end
  end
end

So only when the request has a subdomain and the subdomain is not www, then it will match merchants’ routes.

Warden and Devise

Warden is a Rack middleware which provides authentication for web applications. You can register Strategies in Warden to define how to authenticate a user.

Strategies

For example, the following is to add a :password strategy in Warden

1
2
3
4
5
6
7
8
9
10
11

Warden::Strategies.add(:password) do

  def valid?
    params['username'] || params['password']
  end

  def authenticate!
    u = User.authenticate(params['username'], params['password'])
    u.nil? ? fail!("Could not log in") : success!(u)
  end
end

When register a strategy, you should provide a name(:password as above), and an object or block, the object should implement two methods,

• valid?: It’s optional to declare a valid? method, and if you don’t declare it, the strategy will always be run. If you do declare it though, the strategy will only be tried if #valid? evaluates to true. This could be used to check if all necessary parameters are valid in the request

• authenticate!: This is where the work of actually authenticating the request steps in. Here’s where the logic for authenticating your requests occurs.

You have a number of request related methods available.

• request # Rack::Request object
• session # The session object for the request
• params # The parameters of the request
• env # The Rack env object

There are also a number of actions you can take in your strategy.

• halt! # halts cascading of strategies. Makes this one the last one processed
• pass # ignore this strategy. It is not required to call this and this is only sugar
• success! # Supply success! with a user object to log in a user. Causes a halt!
• fail! # Sets the strategy to fail. Causes a halt!
• redirect! # redirect to another url. You can supply it with params to be encoded and also options. Causes a halt!
• custom! # return a custom rack array to be handed back untouched. Causes a halt! There’s a couple of misc things to do too:

headers # set headers to respond with relevant to the strategy errors # provides access to an errors object. Here you can put in errors relating to authentication

For more information check the Warden documents.

Scope

After configuring Strategies in Warden, you can set the strategies to scopes. Warden can use different strategies on different scopes, and the scopes are independent and not interfere each other. For example, you can configure two scopes :user and :admin, some resources are protected in :user scope and other advanced resources are protected in :admin scope. If you authenticated :user scope, it still needs authentication for :admin scope for advanced resources. For more details check Warden Wiki

Devise

Devise depends on Warden, and it registers a Strategy database_authenticatable by default,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

module Devise
  module Strategies
    # Default strategy for signing in a user, based on their email and password in the database.
    class DatabaseAuthenticatable < Authenticatable
      def authenticate!
        resource  = valid_password? && mapping.to.find_for_database_authentication(authentication_hash)
        encrypted = false

        if validate(resource){ encrypted = true; resource.valid_password?(password) }
          resource.after_database_authentication
          success!(resource)
        end

        mapping.to.new.password = password if !encrypted && Devise.paranoid
        fail(:not_found_in_database) unless resource
      end
    end
  end
end

Warden::Strategies.add(:database_authenticatable,
Devise::Strategies::DatabaseAuthenticatable)

The Devise::Strategies::DatabaseAuthenticatable extends from Devise::Strategies::Authentiatable, which provides some methods for common authenticate behavior. If you are interested you could check Devise’s source code. In above code, we can see that firstly it finds the resource from authentication_hash, for our case, the resource will be a User instance, and then if validate resource’s password successfully, it calls success!, otherwise it calls fail.

Customize Devise Strategy

So for our case, we should not only check if the user’s username and password are valid, but also we must check if the user is an owner of the merchant.

Let’s create class called Devise::Strategies::DatabaseAuthenticatableForMerchantOwner, we put this class in lib folder,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

module Devise
  module Strategies

    class DatabaseAuthenticatableForMerchantOwner < Authenticatable

      # This code is mostly copied from Devise, but except the authentication of devise
      # the class includes this module should define a method 'custom_validate(resource)' to
      # provide other validates, for example if the resource is admin, or if th resource is 
      # an owner of a merchant etc..
      def authenticate!
        resource  = valid_password? && mapping.to.find_for_database_authentication(authentication_hash)
        encrypted = false

        if validate(resource){ encrypted = true; resource.valid_password?(password) } &&
          custom_validate(resource)
            resource.after_database_authentication
            success!(resource)
        else
          mapping.to.new.password = password if !encrypted && Devise.paranoid
          fail(:not_found_in_database)
          halt!
        end
      end

      def custom_validate(resource)
        merchant = get_merchant
        return false unless merchant.present?
        merchant.owner?(resource)
      end

      def get_merchant
        Merchant.find_by(subdomain: request.subdomain)
      end

    end
  end
end

The authenticate! method is almost copied from Devise::Strategies::DatabaseAuthenticatable, however, after validate(resource) it also calls method custom_validate, and in custom_validate it will get the merchant from the request’s subdomain and test if the resource(which is a user) is the merchant’s owner.

PS: It’s not a good practice to copy devise souce code here, I will update this blog if I find a better way.

Register our new strategy

Now we need to register this new strategy. After we installed the Devise gem, it will create a file config/initializers/devise.rb, so let’s update this file,

1
2
3
4
5
6
7
8
9
10

config.warden do |manager|

  require 'devise/strategies/database_authenticatable_for_merchant_owner'
  Warden::Strategies.add(:database_authenticatable_for_merchant_owner,
    Devise::Strategies::DatabaseAuthenticatableForMerchantOwner)

  manager.default_strategies(:scope => :user).delete :database_authenticatable
  manager.default_strategies(:scope => :user).push :database_authenticatable_for_merchant_owner

end

Here we need to require the file explicitly. And then we add the strategy :database_authenticatable_for_merchant_owner.

And in the routes.rb, when we call

1
2
3

Rails.application.routes.draw do
  devise_for :users
end

Devise will define a scope :user in Warden, and it will add the :database_authenticatable in the default_strategies list. Since we want to use :database_authenticatable_for_merchant_owner strategy, we delete :database_authenticatable and push :database_authenticatable_for_merchant_owner

After this, our application will use :database_authenticatable_for_merchant_owner and support Multitenancy authentication.

If you want to learn more about Rails and Devise, this page at udemy provides a good list of some Rails tutorials and can be used as a reference guide.

Long time no see! In last part we have shown that the template is rendered through ActionView::Renderer#render method, so in this part we will check this class in detail.

Now let’s check this class,

ActionView::Renderer

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

module ActionView

  # This is the main entry point for rendering. It basically delegates
  # to other objects like TemplateRenderer and PartialRenderer which
  # actually renders the template.
  #
  # The Renderer will parse the options from the +render+ or +render_body+
  # method and render a partial or a template based on the options. The
  # +TemplateRenderer+ and +PartialRenderer+ objects are wrappers which do all
  # the setup and logic necessary to render a view and a new object is created
  # each time +render+ is called.
  class Renderer
    attr_accessor :lookup_context

    def initialize(lookup_context)
      @lookup_context = lookup_context
    end

    # Main render entry point shared by AV and AC.
    def render(context, options)
      if options.key?(:partial)
        render_partial(context, options)
      else
        render_template(context, options)
      end
    end

    # Direct accessor to template rendering.
    def render_template(context, options) #:nodoc:
      TemplateRenderer.new(@lookup_context).render(context, options)
    end
  end
end

This class is initialized with a lookup_context, which is of class ActionView::LookupContext. And when the render method is called, for a normal template, the options has no :partial key, so render_template is called. We can see that the render_template method is actually creates an instance of TemplateRenderer and calls its render method.

So let’s have a look at this TemplateRenderer class,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

module ActionView

  class TemplateRenderer < AbstractRenderer

    def render(context, options)
      @view    = context
      @details = extract_details(options)
      template = determine_template(options)

      prepend_formats(template.formats)

      @lookup_context.rendered_format ||= (template.formats.first || formats.first)

      render_template(template, options[:layout], options[:locals])
    end

    private

    # Determine the template to be rendered using the given options.
    def determine_template(options)
      keys = options.has_key?(:locals) ? options[:locals].keys : []

      if options.key?(:body)
        Template::Text.new(options[:body])
      elsif options.key?(:text)
        Template::Text.new(options[:text], formats.first)
      elsif options.key?(:plain)
        Template::Text.new(options[:plain])
      elsif options.key?(:html)
        Template::HTML.new(options[:html], formats.first)
      elsif options.key?(:file)
        with_fallbacks { find_template(options[:file], nil, false, keys, @details) }
      elsif options.key?(:inline)
        handler = Template.handler_for_extension(options[:type] || "erb")
        Template.new(options[:inline], "inline template", handler, :locals => keys)
      elsif options.key?(:template)
        if options[:template].respond_to?(:render)
          options[:template]
        else
          find_template(options[:template], options[:prefixes], false, keys, @details)
        end
      else
        raise ArgumentError, "You invoked render but did not give any of :partial, :template, :inline, :file, :plain, :text or :body option."
      end
    end
  end
end

This class extends from ActionView::AbstractRenderer, and the template is determined by the determinate_template method, we can see that this method checks some keys in options hash, for example :body, :text, etc. And last one it checks the :template key. From Part 1 we already knew that if the :template key is not set explicitly, the options[:template] will be the action name. And options[:prefixes] is an array. For example, when the index action of ArticlesController is accessed, the :template and :prefixes will be,

• :prefixes : array [“articles”, “application”]
• :template : string “index”

So the template is found by calling find_template method, and this method is defined in super class ActionView::AbstractRenderer, and let’s have a look at this class.

ActionView::AbstractRenderer

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

module ActionView

  class AbstractRenderer

    delegate :find_template, :template_exists?, :with_fallbacks, :with_layout_format, :formats, :to => :@lookup_context


    def initialize(lookup_context)
      @lookup_context = lookup_context
    end

    def render
      raise NotImplementedError
    end

    protected

    def extract_details(options)
      @lookup_context.registered_details.each_with_object({}) do |key, details|
        value = options[key]

        details[key] = Array(value) if value
      end
    end
  end

end

We can see that the find_template method actually is delegated to lookup_context, we will check this method later, but remember that when find_template method is called, it is passed a details object, which is returned by the extract_details method.

Check the extract_details method above, we can see that this method returns a hash. It get the registered_details from the lookup_context and then check if the options contains the key, if yes, the result hush will be the options value for that key(tranformed to an array).

So what’s in this ActionView::LookupContext#registered_details? Let’s check that.

ActionView::LookupContext#registered_details

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

module ActionView

  # LookupContext is the object responsible to hold all information required to lookup
  # templates, i.e. view paths and details. The LookupContext is also responsible to
  # generate a key, given to view paths, used in the resolver cache lookup. Since
  # this key is generated just once during the request, it speeds up all cache accesses.
  class LookupContext #:nodoc:

    mattr_accessor :registered_details
    self.registered_details = []

    def self.register_detail(name, options = {}, &block)
      self.registered_details << name
      initialize = registered_details.map { |n| "@details[:#{n}] = details[:#{n}] || default_#{n}" }

      Accessors.send :define_method, :"default_#{name}", &block
      Accessors.module_eval <<-METHOD, __FILE__, __LINE__ + 1
        def #{name}
          @details.fetch(:#{name}, [])
        end

        def #{name}=(value)
          value = value.present? ? Array(value) : default_#{name}
          _set_detail(:#{name}, value) if value != @details[:#{name}]
        end

        remove_possible_method :initialize_details
        def initialize_details(details)
          #{initialize.join("\n")}
        end
      METHOD
    end

    # Holds accessors for the registered details.
    module Accessors #:nodoc:
    end

    register_detail(:locale) do
      locales = [I18n.locale]
      locales.concat(I18n.fallbacks[I18n.locale]) if I18n.respond_to? :fallbacks
      locales << I18n.default_locale
      locales.uniq!
      locales
    end

    register_detail(:formats) { ActionView::Base.default_formats || [:html, :text, :js, :css,  :xml, :json] }
    register_detail(:variants) { [] }
    register_detail(:handlers){ Template::Handlers.extensions }
  end

end

We can see that it has a module attribute registered_details, which is an array, whose elements is the detail name.

For example, in a rails console we can call ActionView::LookupContext.registered_details to see the default detail keys.

We can see that by default it registered 4 details: :locale, :formats, :variants and :handlers, just as in above code, the register_detail is called 4 times and passed those details keys one by one.

And also the class maintains an instance variable called @details which is a hash. The key of this hash is the detail key, the value normally is an array which is the detail value initialized from initialize_details. (We will talk about initialize_details later)

Each time the register_detail is called, it will add several instance methods to the class. For example, when passed :locale to the method, it will define following methods,

• default_locale : returns the default locale which is the result returned from the passed block.
• locale: which returns the locale value stored in instance variable @details
• locale=: which sets the locale value to @details.

And also each time the registered_detail is called, the initialize_details will be redefined. For example, when firstly calling register_detail(:locale), the initialize_details will be like following,

1
2
3
4
5
6
7
8
9
10

module ActionView

  class LookupContext #:nodoc:

    def initialize_details(details)
      @details[:locale] = details[:locale] || default_locale
    end
  end

end

And then when register_detail(:formats) is called, the initialize_details will be redefined like following,

1
2
3
4
5
6
7
8
9
10
11

module ActionView

  class LookupContext #:nodoc:

    def initialize_details(details)
      @details[:locale] = details[:locale] || default_locale
      @details[:formats] = details[:formats] || default_formats
    end
  end

end

So the initialize_details is just to get the details values from details for each registered detail key, if the key is not found, it will set the default detail values.

ActionView::LookupContext#find_template

Now let’s check the ActionView::LookupContext#find_template method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

module ActionView

  class LookupContext

    include ViewPaths

    # Helpers related to template lookup using the lookup context information.
    module ViewPaths
      attr_reader :view_paths, :html_fallback_for_js

      # Whenever setting view paths, makes a copy so that we can manipulate them in
      # instance objects as we wish.
      def view_paths=(paths)
        @view_paths = ActionView::PathSet.new(Array(paths))
      end

      def find(name, prefixes = [], partial = false, keys = [], options = {})
        @view_paths.find(*args_for_lookup(name, prefixes, partial, keys, options))
      end
      alias :find_template :find

      def find_all(name, prefixes = [], partial = false, keys = [], options = {})
        @view_paths.find_all(*args_for_lookup(name, prefixes, partial, keys, options))
      end

      def exists?(name, prefixes = [], partial = false, keys = [], options = {})
        @view_paths.exists?(*args_for_lookup(name, prefixes, partial, keys, options))
      end
      alias :template_exists? :exists?

      # Adds fallbacks to the view paths. Useful in cases when you are rendering
      # a :file.
      def with_fallbacks
        added_resolvers = 0
        self.class.fallbacks.each do |resolver|
          next if view_paths.include?(resolver)
          view_paths.push(resolver)
          added_resolvers += 1
        end
        yield
      ensure
        added_resolvers.times { view_paths.pop }
      end

    protected

      def args_for_lookup(name, prefixes, partial, keys, details_options) #:nodoc:
        name, prefixes = normalize_name(name, prefixes)
        details, details_key = detail_args_for(details_options)
        [name, prefixes, partial || false, details, details_key, keys]
      end

      # Compute details hash and key according to user options (e.g. passed from #render).
      def detail_args_for(options)
        return @details, details_key if options.empty? # most common path.
        user_details = @details.merge(options)

        if @cache
          details_key = DetailsKey.get(user_details)
        else
          details_key = nil
        end

        [user_details, details_key]
      end

      # Support legacy foo.erb names even though we now ignore .erb
      # as well as incorrectly putting part of the path in the template
      # name instead of the prefix.
      def normalize_name(name, prefixes) #:nodoc:
        prefixes = prefixes.presence
        parts    = name.to_s.split('/')
        parts.shift if parts.first.empty?
        name     = parts.pop

        return name, prefixes || [""] if parts.empty?

        parts    = parts.join('/')
        prefixes = prefixes ? prefixes.map { |p| "#{p}/#{parts}" } : [parts]

        return name, prefixes
      end
    end
  end

end

The find_template is implemented in module ActionView::LookupContext::ViewPaths, and ActionView::LookupContext includes this module. We can see that find_template is just an alias of method find. The find method delegats to @view_paths, which is an instance of ActionView::PathSet. So we need to understand what ActionView::PathSet#find is doing. Actually the ActionView::PathSet#find will delegate the find to a set of PathResolver, which we will check in next part.

In these days there are many free or paid web templates, especially for bootstrap templates. Recently I integrated a Bootstrap template called Loop which is bought on WrapBootstrap. Here I will explain some practices and caveats about the integration.

Location

After extract the template zip file, we can see the organization is like the structure on the left. This template actually contains two templates, one for personal and one for Agency, and what I want is for the Agency. We can see that it puts all resource files like javascripts, CSS files and fonts file in folder assets. I want to keep this structure, so it would be easier to upgrade if it releases new version. The best practice for put 3rd-party template is to put it in vendor/assets, so I create a folder loop under vendor/assets and and copy the assets folder to vendor/assets/loop, like the following figure,

Notice that I changed the assets folder from the original template and named it loop, so when we reference the files under it from browser it will create paths like /assets/loop/js/loop.js because rails will append assets before the resource. This is to make the files in this template not collide with other assets.

Create index file

If you read my blog Using Index Files in Rails 4 Assets Pipeline before, you should know that the index files should be named loop.css.scss and loop.js, under foler vendor/assets/loop.

They have the following contents,

1
2
3
4
5

/*
 *= require loop/css/bootstrap
 *= require loop/css/font-awesome
 *= require loop/css/main
 */

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

//= require loop/js/jquery-2.1.0
//= require loop/js/bootstrap.js
//= require loop/js/plugins/scrollto/jquery.scrollTo-1.4.3.1-min
//= require loop/js/plugins/scrollto/jquery.localscroll-1.2.7-min
//= require loop/js/plugins/easing/jquery.easing.min
//= require loop/js/plugins/google-map/google-map
//= require loop/js/plugins/parallax/jquery.parallax-1.1.3
//= require loop/js/plugins/jpreloader/jpreloader.min
//= require loop/js/plugins/isotope/imagesloaded.pkgd
//= require loop/js/plugins/isotope/isotope.pkgd.min
//= require loop/js/plugins/wow/wow
//= require loop/js/plugins/flexslider/jquery.flexslider-min
//= require loop/js/plugins/magnific/jquery.magnific-popup.min
//= require loop/js/plugins/supersized/supersized.3.2.7.min
//= require loop/js/plugins/supersized/supersized.shutter.min
//= require loop/js/loop.js

Basically the index files include all dependent css and javascripts, but pay atthention that their path should start with loop since we have put all assets in vendor/assets/loop/loop folder.

Update CSS files

In the template’s CSS files, there are reference to images such as background image, like following code,

1
2
3
4
5
6

body.fullscreen-image-background {
  background-image: url('../img/sliders/slider3.png?1393983498');
  background-size: cover;
  background-repeat: no-repeat;
  background-position: center center;
}

Such url won’t work because Rails will precompile those assets in public/assets folder, so we name this file to main.css.scss, and use image-url tag, like following,

1
2
3
4
5
6

body.fullscreen-image-background {
  background-image: image-url('loop/img/sliders/slider3.png');
  background-size: cover;
  background-repeat: no-repeat;
  background-position: center center;
}

Update JS files

Sometimes the JS files also reference image files, like following code in loop.js,

1
2
3
4
5

slides:   [       // Slideshow Images
              {image : 'assets/img/sliders/slider1.png', title : '<div class="hero-text"><h2 class="hero-heading">HANDCRAFTED</h2><p>Built to provide great visitor experience</p></div>', thumb : '', url : ''},
              {image : 'assets/img/sliders/slider2.png', title : '<div class="hero-text"><h2 class="hero-heading">PARALLAX</h2><p>Scrolling the page is fun with parallax background</p></div>', thumb : '', url : ''},
              {image : 'assets/img/sliders/slider3.png', title : '<div class="hero-text"><h2 class="hero-heading">BUY ONE FOR TWO</h2><p>Buy one to get both of the agency and personal theme</p></div>', thumb : '', url : ''}
            ]

For the same reason, after deploy production the JS files can’t find the image files also, in such case we should depend on the asset pipeline and change the file to loop.js.erb, and use image_path.

1
2
3
4
5

slides:   [       // Slideshow Images
              {image : '<%= image_path 'loop/img/sliders/slider1.png'%>', title : '<div class="hero-text"><h2 class="hero-heading">HANDCRAFTED</h2><p>Built to provide great visitor experience</p></div>', thumb : '', url : ''},
              {image : '<%= image_path 'loop/img/sliders/slider2.png'%>', title : '<div class="hero-text"><h2 class="hero-heading">PARALLAX</h2><p>Scrolling the page is fun with parallax background</p></div>', thumb : '', url : ''},
              {image : '<%= image_path 'loop/img/sliders/slider3.png'%>', title : '<div class="hero-text"><h2 class="hero-heading">BUY ONE FOR TWO</h2><p>Buy one to get both of the agency and personal theme</p></div>', thumb : '', url : ''}
            ]

Also in our erb files, if we want to reference image files we should use image_path.

Precompile configuration

By default when we run rake assets:precompile, Rails won’t precompile the assets in vendor. So we need to add the loop template into configuration, in application.rb, we update like following,

1

config.assets.precompile += %w(loop.css loop.js loop/*)

So it will precompile index files and all assets in vendor/assets/loop/loop folder.

In last part we introduced that when Rails looks for a template, it firstly populate an options hash by calling _normalize_render. And then in the render method, it will call render_to_body and pass the options, like the following code,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

module AbstractController

  module Rendering

    # Normalize arguments, options and then delegates render_to_body and
    # sticks the result in self.response_body.
    # :api: public
    def render(*args, &block)
      options = _normalize_render(*args, &block)
      self.response_body = render_to_body(options)
      _process_format(rendered_format, options) if rendered_format
      self.response_body
    end
  end

end

The render_to_body will select the templated based on the values in options hash. If we check the source code of AbstractController::Rendering#render_to_body, it’s nothing. Just as usual, it’s overridden by other modules.

1
2
3
4
5
6
7
8
9
10
11
12

module AbstractController

  module Rendering

    # Performs the actual template rendering.
    # :api: public
    def render_to_body(options = {})
    end

  end

end

Just as last part, we could run ApplicationController.ancestors_that_implement_instance_method to find what classes or modules implement that method, and we will find the following,

1
2

2.0.0-p353 :008 > ApplicationController.ancestors_that_implement_instance_method(:render_to_body)
 => [ActionController::Renderers, ActionController::Rendering, ActionView::Rendering, AbstractController::Rendering]

We can see three modules implement that method: ActionController::Renderers, ActionController::Rendering, and ActionView::Rendering. Let’s look at each of them one by one.

ActionController::Renderers#render_to_body

For ActionController::Renderers#render_to_body method, it registers a set of renderers, and then if the options contains the renderer key, then it will call that renderer. If no renderer is found, it just call super

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

module ActionController

  module Renderers

    def render_to_body(options)
      _render_to_body_with_renderer(options) || super
    end

    def _render_to_body_with_renderer(options)
      _renderers.each do |name|
        if options.key?(name)
          _process_options(options)
          method_name = Renderers._render_with_renderer_method_name(name)
          return send(method_name, options.delete(name), options)
        end
      end
      nil
    end

  end

end

This is mainly for calling render and pass parameters like :json, :xml, like the following code,

1
2
3
4
5
6
7

class ArticlesController < ApplicationController

  def index
    @articles = Articles.all
    render json: @articles
  end
end

Since :json is a registered renderer in ActionController::Renderers, it will call that renderer. You can also register your own renderer by calling ActionController::Renderers.add.

ActionController::Rendering#render_to_body

If in ActionController::Renderers#render_to_body, it doesn’t find a renderer, then it will call super, which is ActionController::Rendering#render_to_body. Let’s look at what this module does in the method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

module ActionController

  module Rendering

    RENDER_FORMATS_IN_PRIORITY = [:body, :text, :plain, :html]

    def render_to_body(options = {})
      super || _render_in_priorities(options) || ' '
    end

    private

    def _render_in_priorities(options)
      RENDER_FORMATS_IN_PRIORITY.each do |format|
        return options[format] if options.key?(format)
      end

      nil
    end

  end

end

Notice that this method calls super first, it only call _render_in_priorities if super returns nothing.

In _render_in_priorities it searches the RENDER_FORMATS_IN_PRIORITY one by one, and return the option value if it finds the format.

In this module when it calls super, it is calling ActionView::Rendering#render_to_body method. Let’s have a look.

ActionView::Rendering#render_to_body

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

module ActionView

  module Rendering

    def render_to_body(options = {})
      _process_options(options)
      _render_template(options)
    end

    # Returns an object that is able to render templates.
    # :api: private
    def view_renderer
      @_view_renderer ||= ActionView::Renderer.new(lookup_context)
    end

    private

      # Find and render a template based on the options given.
      # :api: private
      def _render_template(options) #:nodoc:
        variant = options[:variant]

        lookup_context.rendered_format = nil if options[:formats]
        lookup_context.variants = variant if variant

        view_renderer.render(view_context, options)
      end

  end

end

It turns out that here is the the meat we are looking for. The render_to_body calls _render_template, and for the _render_template, it calls view_renderer.render(view_context, options).

The view_renderer is an instance of ActionView::Renderer, and when it’s initialized, it’s passing a lookup_context object, which is an instance of ActionView::LookupContext. The ActionView::LookupContext contains all information about looking for a template based on the options. So in next part we will check this class in detail, and check how LookupContext, ViewPaths, PathSet work together to find the template.

Have you ever wondered for a Rails application, when you access an action in a controller, how Rails finds the template to render? For example, when the action index in ArticlesController is accessed, by default the template app/views/articles/index.html.erb will be selected and rendered. Recently I’m digging the source code of Rails, and I invite you to walk through some source code in ActionPack and ActionView with me. And then I will show how the Rubygem versioncake works by modifying the Rails configuration.

In this first part we firstly check how the render works. Notice that we check Rails 4.2 source code. If you look at another version, the implementation may be slightly different.

The entry point for the render is from the AbstractController::Rendering#render method. The AbstractController is a module shared by ActionController and ActionMailer. Since these two modules share a lot of functionalities, Rails extract those same functionalities into AbstractController. Let’s have a look at this method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

module AbstractController

  module Rendering

    # Normalize arguments, options and then delegates render_to_body and
    # sticks the result in self.response_body.
    # :api: public
    def render(*args, &block)
      options = _normalize_render(*args, &block)
      self.response_body = render_to_body(options)
      _process_format(rendered_format, options) if rendered_format
      self.response_body
    end
  end

end

In our controller, we could call render method directly. For example, we can call render ‘new’ to render the new.html.erb template. Or if we don’t call render explicitly, there is a module ActionController::ImplicitRender which will call a default render.

1
2
3
4
5
6
7
8
9
10
11
12
13

module ActionController
  module ImplicitRender
    def send_action(method, *args)
      ret = super
      default_render unless performed?
      ret
    end

    def default_render(*args)
      render(*args)
    end
  end
end

The send_action will be called when the action of a controller is triggered. It first calls super, then if in the action it doesn’t render anything, the performed? will return false. So default_render is called. We can see that when call default_render it just calls render without any arguments.

In AbstractController::Rendering#render method, it firstly calls _normalize_render then calls render_to_body. The _normalize_render returns an options object which is a Hash. In this part we will examine the _normalize_render method to see how the options is generated.

Let’s see how _normalize_render is implemented.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

module AbstractController

  module Rendering

    private

    # Normalize args and options.
    # :api: private
    def _normalize_render(*args, &block)
      options = _normalize_args(*args, &block)
      #TODO: remove defined? when we restore AP <=> AV dependency
      if defined?(request) && request && request.variant.present?
        options[:variant] = request.variant
      end
      _normalize_options(options)
      options
    end

  end

end

We see that it calls _normalize_args and _normalize_options methods. The _normalize_args and _normalize_options have different purposes.

_normalize_args

The _normalize_args is to convert all args into an options hash. For example when we call render method, we could call like this,

1

render 'new', status: :ok

Here the first argument ‘new’ is a String, and _normalize_args is responsible to put this first argument in options hash and give it an appropriate key.

Let’s see how it’s implemented.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

module AbstractController

  module Rendering

    # Normalize args by converting render "foo" to render :action => "foo" and
    # render "foo/bar" to render :file => "foo/bar".
    # :api: plugin
    def _normalize_args(action=nil, options={})
      if action.is_a? Hash
        action
      else
        options
      end
    end

  end

end

We see that this method by default almost does nothing, if the action is a Hash, it returns action, otherwise if action is for example, a string, it returns the second parameter which is the options hash.

Notice that ApplicationController includes some other modules which override this method, as we will see later.

_normalize_options

The _normalize_options method is for the modules to include other options. For a Rails application, the ApplicationController extends from ActionController::Base, and ActionController::Base includes a lot of modules, each module could override this method and add some other options.

Let’s firstly check how this method is implemented in AbstractController::Rendering.

1
2
3
4
5
6
7
8
9
10
11
12
13

module AbstractController

  module Rendering

    # Normalize options.
    # :api: plugin
    def _normalize_options(options)
      options
    end

  end

end

By default this method does nothing. But it will be overridden in other modules.

Override _normalize_args

Rails source code is complex, one reason is because there are many modules could override other modules’s methods. For example, in a ArticlesController, let’s see its ancestors,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

$ rails c
Loading development environment (Rails 4.2.0)
2.0.0-p353 :001 > puts ArticlesController.ancestors
ArticlesController
#<Module:0x007f8f0a971800>
ApplicationController
#<Module:0x007f8f0a8f93c8>
#<Module:0x007f8f05465118>
#<Module:0x007f8f05465140>
ActionController::Base
WebConsole::ControllerHelpers
Turbolinks::XHRHeaders
Turbolinks::Cookies
Turbolinks::XDomainBlocker
Turbolinks::Redirection
ActiveRecord::Railties::ControllerRuntime
ActionDispatch::Routing::RouteSet::MountedHelpers
ActionController::ParamsWrapper
ActionController::Instrumentation
ActionController::Rescue
ActionController::HttpAuthentication::Token::ControllerMethods
ActionController::HttpAuthentication::Digest::ControllerMethods
ActionController::HttpAuthentication::Basic::ControllerMethods
ActionController::DataStreaming
ActionController::Streaming
ActionController::ForceSSL
ActionController::RequestForgeryProtection
ActionController::Flash
ActionController::Cookies
ActionController::StrongParameters
ActiveSupport::Rescuable
ActionController::ImplicitRender
ActionController::MimeResponds
ActionController::Caching
ActionController::Caching::Fragments
ActionController::Caching::ConfigMethods
AbstractController::Callbacks
ActiveSupport::Callbacks
ActionController::EtagWithTemplateDigest
ActionController::ConditionalGet
ActionController::Head
ActionController::Renderers::All
ActionController::Renderers
ActionController::Rendering
ActionView::Layouts
ActionView::Rendering
ActionController::Redirecting
ActionController::RackDelegation
ActiveSupport::Benchmarkable
AbstractController::Logger
ActionController::UrlFor
AbstractController::UrlFor
ActionDispatch::Routing::UrlFor
ActionDispatch::Routing::PolymorphicRoutes
ActionController::ModelNaming
ActionController::HideActions
ActionController::Helpers
AbstractController::Helpers
AbstractController::AssetPaths
AbstractController::Translation
AbstractController::Rendering
ActionView::ViewPaths
ActionController::Metal
AbstractController::Base
ActiveSupport::Configurable
Object
ActiveSupport::Dependencies::Loadable
PP::ObjectMixin
JSON::Ext::Generator::GeneratorMethods::Object
Kernel
BasicObject

We can see that for a typical controller, it has a lot of ancestors and most of them are modules. All modules after AbstractController::Rendering could override its methods. I have created a gist to check which ancestors implement a method.

1
2
3
4
5
6
7

class Module
  def ancestors_that_implement_instance_method(instance_method)
    ancestors.find_all do |ancestor|
      (ancestor.instance_methods(false) + ancestor.private_instance_methods(false)).include?(instance_method)
    end
  end
end

Run the above code in a rails console, then you can call ClassName.ancestors_that_implement_instance_method to check what ancestors implement a method.

Let’s first see what ancestors override the _normalize_args method,

1
2
3
4

$ rails c
Loading development environment (Rails 4.2.0)
2.0.0-p353 :013 >   ArticlesController.ancestors_that_implement_instance_method(:_normalize_args)
 => [ActionController::Rendering, ActionView::Rendering, AbstractController::Rendering]

Two modules override this instance method: ActionView::Rendering and ActionController::Rendering. Let’s look them in order from top to down.

Let’s look at ActionView::Rendering first,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

module ActionView
  module Rendering
    # Normalize args by converting render "foo" to render :action => "foo" and
    # render "foo/bar" to render :template => "foo/bar".
    # :api: private
    def _normalize_args(action=nil, options={})
      options = super(action, options)
      case action
      when NilClass
      when Hash
        options = action
      when String, Symbol
        action = action.to_s
        key = action.include?(?/) ? :template : :action
        options[key] = action
      else
        options[:partial] = action
      end

      options
    end
  end
end

We can see that for the first argument action, if it’s a string and the string include ‘/’, the key for this argument is :file, and if it doesn’t include ‘/’, the key is :action.

So if we call render ‘new’, the options will be { action: ‘new’ }, if we call render ‘articles/new’, the options will be { file: ‘articles/new’ }

Now let’s see how ActionController::Rendering overrides this method

1
2
3
4
5
6
7
8
9
10
11

module ActionController
  module Rendering
    # Normalize arguments by catching blocks and setting them on :update.
    def _normalize_args(action=nil, options={}, &blk) #:nodoc:
      options = super
      options[:update] = blk if block_given?
      options
    end
  end
end

We can see that for this override, if the method is passed a block, it will set the block to options[:update]

Override _normalize_options

Just like _normalize_args, let’s examine what modules override _normalize_options. We can see following modules implement _normalize_options: [ActionController::Rendering, ActionView::Layouts, ActionView::Rendering, AbstractController::Rendering].

Let’s check ActionView::Rendering first,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

module ActionView
  module Rendering

    # Normalize options.
    # :api: private
    def _normalize_options(options)
      options = super(options)
      if options[:partial] == true
        options[:partial] = action_name
      end

      if (options.keys & [:partial, :file, :template]).empty?
        options[:prefixes] ||= _prefixes
      end

      options[:template] ||= (options[:action] || action_name).to_s
      options
    end
  end
end

We can see that it addes three options by default,

• If the options[:partial] is true, then it will set options[:partial] to action_name, the action_name is just the name of the action that is triggered. For example, if the index action of ArticlesController is triggered, action_name will be index.
• If the options doesn’t include :partial, :file or :template, it set options[:prefixes]. Let’s see what’s set for this :prefixes in a minute.
• It set the options[:template]. It’s either passed from the arguments or just use the action_name.

So what’s in options[:prefixes], let’s see how _prefixes method is implemented.

The AbstractController::Rendering module includes ActionView::ViewPaths module. And _prefixes method is implemented there.

ActionView::ViewPaths is an important module which we will examine in more details in later parts. It’s to manage the view paths for controllers. For example, by default Rails will append a view path #{Rails.root}app/views so the application knows to search templates in that specific view path.

For now let’s just focus on _prefixes method.

1
2
3
4
5
6
7
8
9
10
11
12

module ActionView

  module ViewPaths

    # The prefixes used in render "foo" shortcuts.
    def _prefixes # :nodoc:
      self.class._prefixes
    end

  end

end

The _prefixes just calls the class method _prefixes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

module ActionView

  module ViewPaths
    module ClassMethods
      def _prefixes # :nodoc:
        @_prefixes ||= begin
          return local_prefixes if superclass.abstract?

          local_prefixes + superclass._prefixes
        end
      end

      private

      # Override this method in your controller if you want to change paths prefixes for finding views.
      # Prefixes defined here will still be added to parents' <tt>._prefixes</tt>.
      def local_prefixes
        [controller_path]
      end

    end
  end

end

Notice that Rails 4.2 implements that method and handles some deprecated parent prefixes. In above code I omitted that handling for clarity.

This ActionView::ViewPaths._prefixes calls recursively. it append local_prefixes with superclass’s _prefixes. The local_prefixes has just one element: controller_path.

The controller_path method is very simple, it’s implemented in AbstractController::Base. For example, for a controller ArticlesController, its controller_path will be articles, and for a controller Articles::CommentsController, its controller_path will be articles/comments

So the _prefixes method first gets it’s parent prefixes, and then prepends the current controller_path to the front.

For example, if our application has a ArticlesController, in our rails controller, we can call the following code to show the _prefixes.

1
2
3
4

$ rails c
Loading development environment (Rails 4.2.0)
2.0.0-p353 :001 > ArticlesController.new.send(:_prefixes)
 => ["articles", "application"]

We see that the prefixes contains two elements: articles and application. It’s because the ApplicationController extends from ActionController::Base, but ActionController::Base is abstract.

So now we can see that for ArticlesController#index action, when we just call render without any arguments, the options will contains following elements,

• :prefixes : array [“articles”, “application”]
• :template : string “index”

Now let’s see next one, how ActionView::Layouts overrides _normalize_options method,

1
2
3
4
5
6
7
8
9
10
11
12
13
14

module ActionView
  module Layouts

    def _normalize_options(options) # :nodoc:
      super

      if _include_layout?(options)
        layout = options.delete(:layout) { :default }
        options[:layout] = _layout_for_option(layout)
      end
    end

  end
end

We can see that if the options[:layout] is not set, the default layout is :default. And then options[:layout] is set to the result returned by _layout_for_option.

If you are interested you could check how _layout_for_options is implemented. When this module searches for the layout it will search “app/views/layouts/#{class_name.underscore}.rb” first, if it doesn’t found, then it will search super class. Since when a rails application is generated, an application.html.erb will be put in app/views/layouts and since by default all controllers’ parent is ApplicationController, so by default this layout will be used.

Finally let’s see how ActionController::Rendering overrides _normalize_options

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

module ActionController
  module Rendering

    # Normalize both text and status options.
    def _normalize_options(options) #:nodoc:
      _normalize_text(options)

      if options[:html]
        options[:html] = ERB::Util.html_escape(options[:html])
      end

      if options.delete(:nothing)
        options[:body] = nil
      end

      if options[:status]
        options[:status] = Rack::Utils.status_code(options[:status])
      end

      super
    end

  end
end

So this method just process :html, :nothing and :status options, which is straight forward.

So finally let’s see when we call render in ArticlesController#index without any arguments, the options will contains following values,

• :prefixes : array [“articles”, “application”]
• :template : string “index”
• :layout : A Proc when called will return “app/views/layouts/application.html.erb”

Now we know how the options is normalized. And when Rails determines which template to render, it will extract details from the options. And we will look at how Rails determines template in later parts.

Here is the Part 3 of Deploying Rails Application on Ubuntu 14.04, and also the final part. In this part we will explain what the Capistrano template does.

Capistrano is a remote server automation and deployment tool written in Ruby. Capistrano 3 extends Rake DSL with its own set of DSL. I recommend you read the documents on Capistrano website.

Install Capistrano

In Gemfile we add the following gems,

1
2
3
4
5
6
7
8
9
10

gem 'capistrano', '~> 3.1.0', group: :development

# rails specific capistrano funcitons
gem 'capistrano-rails', '~> 1.1.0', group: :development

# integrate bundler with capistrano
gem 'capistrano-bundler', group: :development

# if you are using RBENV
gem 'capistrano-rbenv', "~> 2.0", group: :development

The capistrano-rails, capistrano-bundler, and capistrano-rbenv are capistrano plugins. Let’s focus on capistrano first.

After running bundle install, if we run bundle exec cap install, it will generate following folders and files, actually when we copy the files from the capistrano template in part 2, it does the same thing.

1
2
3
4
5
6
7
8
9

├──  Capfile
├── config
│   ├── deploy
│   │   ├── production.rb
│   │   └── staging.rb
│   └── deploy.rb
└── lib
    └── capistrano
            └── tasks

• In Capfile, It requires all necessary files, and also it includes all customized tasks in lib/capistrano/tasks.
• Normally you have a production server and a staging server. So before you deploy new features to production server, you want to deploy to staging server first and after all QA are passed. So in config/deploy.rb, it defines the parameters that apply to all environments, and in config/deploy/production.rb and config/deploy/staging.rb it defines the parameters that apply to the specific environment. When we run bundle exec cap production deploy, it will load config/deploy.rb first, then config/deploy/production.rb, and then run the deploy rake task.

Understanding Capistrano

To understand Capistrano, the best way is to read its source code. Actually it’s not that hard as the source code of Capistrano is pretty easy to understand.

DSL

Capistrano defines a DSL. For those interested, can check the capistrano source code in folder lib/capistrano/dsl

Set attributes

You can set some attributes by using set, and then later use fetch to get the attribute value by key. In our sample application config/deploy.rb, you can see it sets a lot of attributes.

The following code snippets illustrate its use.

1
2
3
4
5
6
7
8

set :application, 'deploy_sample'
set :scm, :git

fetch :application # if :application key doesn't exist, it will return nil

fetch :application, 'application_name' # if the :application key doesn't exist, it will return default value 'application_name'

set_if_empty :rbenv_type, :user # set only the :rbenv_type key is not defined

Server and Roles

Think about our deployment, at least we need a database server, an application server which is unicorn, and a web server which is Nginx or Apache. Sometimes db, app and web are on the same server, sometimes they are installed on different servers. Capistrano defines these as roles. For example, in our sample application, we define one server which has three roles,

1

server '192.168.22.10', user: 'deploy', roles: %w{web app db}, primary: true

So later if we want to want to do something on the server that acts as web and app role, we could use roles or release_roles method,

For example, the following code will create directory on the server that act as web and app role,

1
2
3
4
5

task :directories do
  on release_roles :web, :app do
    execute :mkdir, '-p', shared_path, releases_path
  end
end

If the web and app role point to two different servers, each server will create the directory. If we use :all, it will get all servers.

Rake tasks

Capistrano 3 actually is an extension of Rake tasks. So when we run bundle exec cap production deploy, it will run the deploy Rake task.

Let’s have a look at the definition of the deploy task. It’s in Capistrano source code /lib/capistrano/tasks/framework.rake

1
2
3
4
5
6
7
8
9
10

desc 'Deploy a new release.'
task :deploy do
  set(:deploying, true)
  %w{ starting started
      updating updated
      publishing published
      finishing finished }.each do |task|
    invoke "deploy:#{task}"
  end
end

So we can see that the deploy task runs the following 8 tasks in sequence, * deploy:starting * deploy:started * deploy:updating * deploy:updated * deploy:publishing * deploy:published * deploy:finishing * deploy:finished

If you are interested you should check the source code of those tasks. For example, in task deploy:updating it will invoke deploy:symlink:shared, which in turns invokes deploy:symlink:link_files, let’s see how this task is implemented

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

task :deploy do
  task :symlink do

    desc 'Symlink linked files'
    task :linked_files do
      next unless any? :linked_files
      on release_roles :all do
        execute :mkdir, '-p', linked_file_dirs(release_path)

        fetch(:linked_files).each do |file|
          target = release_path.join(file)
          source = shared_path.join(file)
          unless test "[ -L #{target} ]"
            if test "[ -f #{target} ]"
              execute :rm, target
            end
            execute :ln, '-s', source, target
          end
        end
      end
    end
  end
end

We can see that it will fetch the linked_files property, for our deploy sample application, we have set it to %w{config/database.yml config/secrets.yml}, so for each file, it will created a symbolic link in release_path, which points to the file in shared_path, where is release_path? Let’s have a look at Capistrano source code /lib/capistrano/dsl/paths.rb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

def deploy_to
  fetch(:deploy_to)
end

def deploy_path
  Pathname.new(deploy_to)
end

def current_path
  deploy_path.join('current')
end

def release_path
  fetch(:release_path, current_path)
end