In my application I will use Service Account, which means that the application doesn’t redirect to the Google login page and asks the user to login, instead the application calls the Content API on behalf of the Service Account. This step is the same as David’s blog, so I’ll copy & paste here shamelessly.
So let’s create a Service Account in Google Developer Console. Go to the Google Developer Console and navigate to the “Credentials” menu under “API’s and Auth”. From there click on “Create new Client ID”:
Select the “Service Account” option:
As soon as you create the Service Account, the system will generate a Client ID, email address, and a public/private key pair. The private key is immediately downloaded to your browser. Save this private key safely somewhere. There is a password that comes with this private key that you will need later. The password is “notasecret” and is the same for every private key that is generated.
You will notice a client ID and email address are also created for the Service Account you just created. Make note of the email address as you will need this later:
This next step is very important and is not called out very clearly in Google’s documentation and is very easy to miss. Just because you can digitally sign JWT’s and Google’s authentication server can verify your application is calling on behalf of the Service Account, it doesn’t mean your application can access just any account’s data behind the API whether that be Google Analytics, Google Merchant Center, Adwords, etc. In this example we are going have our program call the Content API for Shopping to retrieve a product from the corresponding Google Merchant Center account.
Login to the corresponding Google Merchant Center Account and navigate to the “Users” menu under “Settings”. Enter the Service Account email address here and grant it “Standard” access.
To calling Google Content API, we need to add google-api-client Gem into our Gemfile.
1
|
|
And then run bundle install
To calling Google Content API, we need an Google::APIClient instance.
1 2 3 4 5 6 7 8 9 10 |
|
Remember the private key file we downloaded when we create the Service Account? I saved the file in #{Rails.root}/config with file name content_api.p12 (If your repository is an open source repository, you should not put this file in your repository as it’s very sensitive).
On line 1, it load the key from the content_api.p12, and on line 3 it creates an asserter instance. The ENV[‘GOOGLE_CONTENT_API_CLIENT_ID’] is the Client ID as indicated in the following snapshot. Also you should not put this information in your repository if your repository is public. For my project I put it in a yml file and when the application loads, it will set the key/value in the yml file as environment variables.
On line 6 it creates a Google::APIClient instance, and on line 9 it sets the authorization.
On line 10, it calls client.discovered_api to get the content api, we will use this object to dictate which api we will call when we call the api, as will show in following sections.
To call Google Content API, normally we would call the Google::APIClient#execute method. This method accepts a Hash parameter, and the Hash normally contains the following options,
The :api_method indicates which API we would like to call. Normally we set this option by calling the content object we got above. To know which API to call, take a look at the Google Content API reference.
For example, to call the Accounts: insert API, as the figure below, we should set :api_method to content.accounts.insert
The parameters should be a hash, it indicates the path parameters in the request URL, for example, the Accounts: insert API expects a merchantId param, so to call this API, it should set the parameters to { :merchantId => merchant_id }. Notice that the parameter key is camel case, which doesn’t conform to Ruby convention.
If the API contains a body, we should put the body in the :body_object option. The :body_object should be a Hash, what it contains depends on the specific API.
For example, to create an Account with name ‘my_account’, we should set the :body_object as a hash like this,
:body_object => { :name => ‘my_account’ }
When call the execute method, it returns us a response, we should call response.data? to check if the response contains any data. And if the API is failed, we can check response.data[‘error’]. If the API calling is successful, we can check the documentation to see what the response data contains.For example, for Accounts Insert, if successful it will returns a Accounts resource, so we can call response.data[‘id’] to get the ID of the created Account.
Now we give some examples as following. In the examples, the client is a Google::APIClient instance. Normally you should wrap this client instance in a class, and make the methods as instance methods of the class.
To create a sub-Account, and also give an email as the admin, we will call the following API,
1 2 3 4 5 6 |
|
This method will create a new sub-Account under the parent_merchant_id, and set the admin_email as the admin of the created sub-Account.
To delete a sub-Account, call the method like following,
1 2 3 4 |
|
It will delete the sub-Account whose ID is account_id from the parent_merchant_id.
To create a feed, call the API like following,
1 2 3 4 5 6 7 8 |
|
The above method will create a feed which targets Australia and the language is english. And we set the fetch schedule as 8am of the day.
]]>In last part we saw that ActionView::LookupContext#find_template is just a delegation to ActionView::PathSet#find. So let’s check what ActionView::PathSet#find is doing.
From the class name of PathSet we can get an idea that this class is to manage a set of pathes, where the templates are stored. Let’s have a look at the definition of this class.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
|
In LookupContext class, we can see how the ActionView::PathSet instance is created.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
The view_paths= method is defind in module ActionView::LookupContext::ViewPaths and ActionView::LookupContext includes this module. We can see that the when calling view_paths= method, an instance of ActionView::PathSet will be created and set to the @view_paths instance variable of ActionView::LookupContext.
So how this view_paths= method is called? It’s in ActionView::LookupContext#initialize method.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
So the view_paths is passed to ActionView::LookupContext instances in initialize. And in ActionView::ViewPaths it initializes ActionView::LookupContext.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
|
This ActionView::ViewPaths will be included in a controller class. We can see that by default the view_paths is an empty array. But it provides append_view_path and prepend_view_path to add a view path at the end or the front of view paths. Notice that the prepend_view_path and append_view_path are defined both as class methods and instance methods.
So how the view path is initialized? Actually it’s in Rails::Engine class.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
a initializer defines a code block which will be called during application startup. So when load :action_controller it will call prepend_view_path and the view_paths contains only one element, which is the app/views folder in your rails application.
Now understand how ActionView::PathSet is initialized, and we also know that in find method it actually delegates to the resolver. So let’s see how the resolver implements the method.
The Resolver is to resolve a template when passing the details. The class relationship is as the above diagram,
We can see that by default the resolver used in ActionView::PathSet, which is ActionView::OptimizedFileSystemResolver, extends from ActionView::FileSystemResolver, which in terms extends from ActionView::PathResolver, which extends from ActionView::Resolver, which is the base class for all resolvers.
The base class ActionView::Resolver provides some common functionalities such as caching for all sub-class resolvers. However here I will focus on ActionView::PathResolver, which implements the logic how to find a template in a path.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
|
The most important methods are build_query and find_template_paths.
articles/index{.en,}{.html,}{+:variants,}{.haml,}
For this query, in find_template_paths it will call Dir[query], which is actually an alias to glob which is to expand the query. For example, for the above query, {p, q} matchs either p, or q. So for example for {.en,} it could either match .en or an empty string. So if we define a template articles/index.html.haml, it will match the query. For more information, consult Ruby doc.
So for each matched template, in query method it will create a Template to wrap the template file.
So in summary here we can see that the flow for Rails to find a template is as following,
Before I worked in a large company which has its own DBAs. Yes I know SQL,I know Stored Procedures and I think I know something about indexes, but for database performance tuning, we always delegate the task to the DBA. And currently I work in a small company which doesn’t have a specific DBA role, so developers have to be DBA (Full Stack right?). So for learning something about database performance tuning, I bought this book. Oh I really love it and I learned so many things in one week!
This book talks about indexes and only one type of indexes: B-Tree. It firstly starts with the basics of index and the WHERE clause. And then proceed to join, clustering, sorting and grouping, and also partial results and modifying data. After reading this book, you should have a firm understanding how indexes work and I’m sure that this is needed for every Rails developer. As I totally agree the author’s claim, that SQL performance tuning is not just a DBA task but a developer’s task, as the performance is related to business logic and only developers understand that. And also we should think performance from design, and it’s not an afterthought.
]]>Last year I read the book Multitenancy with Rails written by Ryan Bigg. In the book when he implements the authentication for the multi-tenancy, he uses Warden instead of Devise, cause using Devise will need a lot of customization. And Recently I need to implement this multi-tenancy authentication for a Rails application, and I want to use Devise, so I checked the source code of Warden and Devise and found a way to customize Devise to do it.
In my application, I have two models, Merchant and User, one user can be the owner of one or more merchant, so they are a has_and_belongs_to_many relationship.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
|
Each merchant can access its web interface through a subdomain URL. In model Merchant there is a unique attribute subdomain, this will be used for subdomain. For example, one merchant’s subdomain is skywatch, then the URL for this merchant is http://skywatch.xhop.pe . And for the authentication, a user can login this URL only if he is this merchant’s owner.
And in Merchant class we defined an instance method owner? of check if a user is owner of a merchant.
For all controllers related to a merchant, I put the controllers under a module Merchant.
1 2 3 4 5 6 7 8 9 |
|
We set a constraint Constraints::SubdomainRequired to match merchant’s routes.
1 2 3 4 5 6 7 |
|
So only when the request has a subdomain and the subdomain is not www, then it will match merchants’ routes.
Warden is a Rack middleware which provides authentication for web applications. You can register Strategies in Warden to define how to authenticate a user.
For example, the following is to add a :password strategy in Warden
1 2 3 4 5 6 7 8 9 10 11 |
|
When register a strategy, you should provide a name(:password as above), and an object or block, the object should implement two methods,
valid?: It’s optional to declare a valid? method, and if you don’t declare it, the strategy will always be run. If you do declare it though, the strategy will only be tried if #valid? evaluates to true. This could be used to check if all necessary parameters are valid in the request
authenticate!: This is where the work of actually authenticating the request steps in. Here’s where the logic for authenticating your requests occurs.
You have a number of request related methods available.
There are also a number of actions you can take in your strategy.
headers # set headers to respond with relevant to the strategy errors # provides access to an errors object. Here you can put in errors relating to authentication
For more information check the Warden documents.
After configuring Strategies in Warden, you can set the strategies to scopes. Warden can use different strategies on different scopes, and the scopes are independent and not interfere each other. For example, you can configure two scopes :user and :admin, some resources are protected in :user scope and other advanced resources are protected in :admin scope. If you authenticated :user scope, it still needs authentication for :admin scope for advanced resources. For more details check Warden Wiki
Devise depends on Warden, and it registers a Strategy database_authenticatable by default,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
The Devise::Strategies::DatabaseAuthenticatable extends from Devise::Strategies::Authentiatable, which provides some methods for common authenticate behavior. If you are interested you could check Devise’s source code. In above code, we can see that firstly it finds the resource from authentication_hash, for our case, the resource will be a User instance, and then if validate resource’s password successfully, it calls success!, otherwise it calls fail.
So for our case, we should not only check if the user’s username and password are valid, but also we must check if the user is an owner of the merchant.
Let’s create class called Devise::Strategies::DatabaseAuthenticatableForMerchantOwner, we put this class in lib folder,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
|
The authenticate! method is almost copied from Devise::Strategies::DatabaseAuthenticatable, however, after validate(resource) it also calls method custom_validate, and in custom_validate it will get the merchant from the request’s subdomain and test if the resource(which is a user) is the merchant’s owner.
PS: It’s not a good practice to copy devise souce code here, I will update this blog if I find a better way.
Now we need to register this new strategy. After we installed the Devise gem, it will create a file config/initializers/devise.rb, so let’s update this file,
1 2 3 4 5 6 7 8 9 10 |
|
Here we need to require the file explicitly. And then we add the strategy :database_authenticatable_for_merchant_owner.
And in the routes.rb, when we call
1 2 3 |
|
Devise will define a scope :user in Warden, and it will add the :database_authenticatable in the default_strategies list. Since we want to use :database_authenticatable_for_merchant_owner strategy, we delete :database_authenticatable and push :database_authenticatable_for_merchant_owner
After this, our application will use :database_authenticatable_for_merchant_owner and support Multitenancy authentication.
If you want to learn more about Rails and Devise, this page at udemy provides a good list of some Rails tutorials and can be used as a reference guide.
]]>Now let’s check this class,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
|
This class is initialized with a lookup_context, which is of class ActionView::LookupContext. And when the render method is called, for a normal template, the options has no :partial key, so render_template is called. We can see that the render_template method is actually creates an instance of TemplateRenderer and calls its render method.
So let’s have a look at this TemplateRenderer class,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
|
This class extends from ActionView::AbstractRenderer, and the template is determined by the determinate_template method, we can see that this method checks some keys in options hash, for example :body, :text, etc. And last one it checks the :template key. From Part 1 we already knew that if the :template key is not set explicitly, the options[:template] will be the action name. And options[:prefixes] is an array. For example, when the index action of ArticlesController is accessed, the :template and :prefixes will be,
So the template is found by calling find_template method, and this method is defined in super class ActionView::AbstractRenderer, and let’s have a look at this class.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
We can see that the find_template method actually is delegated to lookup_context, we will check this method later, but remember that when find_template method is called, it is passed a details object, which is returned by the extract_details method.
Check the extract_details method above, we can see that this method returns a hash. It get the registered_details from the lookup_context and then check if the options contains the key, if yes, the result hush will be the options value for that key(tranformed to an array).
So what’s in this ActionView::LookupContext#registered_details? Let’s check that.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
|
We can see that it has a module attribute registered_details, which is an array, whose elements is the detail name.
For example, in a rails console we can call ActionView::LookupContext.registered_details to see the default detail keys.
We can see that by default it registered 4 details: :locale, :formats, :variants and :handlers, just as in above code, the register_detail is called 4 times and passed those details keys one by one.
And also the class maintains an instance variable called @details which is a hash. The key of this hash is the detail key, the value normally is an array which is the detail value initialized from initialize_details. (We will talk about initialize_details later)
Each time the register_detail is called, it will add several instance methods to the class. For example, when passed :locale to the method, it will define following methods,
And also each time the registered_detail is called, the initialize_details will be redefined. For example, when firstly calling register_detail(:locale), the initialize_details will be like following,
1 2 3 4 5 6 7 8 9 10 |
|
And then when register_detail(:formats) is called, the initialize_details will be redefined like following,
1 2 3 4 5 6 7 8 9 10 11 |
|
So the initialize_details is just to get the details values from details for each registered detail key, if the key is not found, it will set the default detail values.
Now let’s check the ActionView::LookupContext#find_template method.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
|
The find_template is implemented in module ActionView::LookupContext::ViewPaths, and ActionView::LookupContext includes this module. We can see that find_template is just an alias of method find. The find method delegats to @view_paths, which is an instance of ActionView::PathSet. So we need to understand what ActionView::PathSet#find is doing. Actually the ActionView::PathSet#find will delegate the find to a set of PathResolver, which we will check in next part.
]]>After extract the template zip file, we can see the organization is like the structure on the left. This template actually contains two templates, one for personal and one for Agency, and what I want is for the Agency. We can see that it puts all resource files like javascripts, CSS files and fonts file in folder assets. I want to keep this structure, so it would be easier to upgrade if it releases new version. The best practice for put 3rd-party template is to put it in vendor/assets, so I create a folder loop under vendor/assets and and copy the assets folder to vendor/assets/loop, like the following figure,
Notice that I changed the assets folder from the original template and named it loop, so when we reference the files under it from browser it will create paths like /assets/loop/js/loop.js because rails will append assets before the resource. This is to make the files in this template not collide with other assets.
If you read my blog Using Index Files in Rails 4 Assets Pipeline before, you should know that the index files should be named loop.css.scss and loop.js, under foler vendor/assets/loop.
They have the following contents,
1 2 3 4 5 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
Basically the index files include all dependent css and javascripts, but pay atthention that their path should start with loop since we have put all assets in vendor/assets/loop/loop folder.
In the template’s CSS files, there are reference to images such as background image, like following code,
1 2 3 4 5 6 |
|
Such url won’t work because Rails will precompile those assets in public/assets folder, so we name this file to main.css.scss, and use image-url tag, like following,
1 2 3 4 5 6 |
|
Sometimes the JS files also reference image files, like following code in loop.js,
1 2 3 4 5 |
|
For the same reason, after deploy production the JS files can’t find the image files also, in such case we should depend on the asset pipeline and change the file to loop.js.erb, and use image_path.
1 2 3 4 5 |
|
Also in our erb files, if we want to reference image files we should use image_path.
By default when we run rake assets:precompile, Rails won’t precompile the assets in vendor. So we need to add the loop template into configuration, in application.rb, we update like following,
1
|
|
So it will precompile index files and all assets in vendor/assets/loop/loop folder.
]]>1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
The render_to_body will select the templated based on the values in options hash. If we check the source code of AbstractController::Rendering#render_to_body, it’s nothing. Just as usual, it’s overridden by other modules.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Just as last part, we could run ApplicationController.ancestors_that_implement_instance_method to find what classes or modules implement that method, and we will find the following,
1 2 |
|
We can see three modules implement that method: ActionController::Renderers, ActionController::Rendering, and ActionView::Rendering. Let’s look at each of them one by one.
For ActionController::Renderers#render_to_body method, it registers a set of renderers, and then if the options contains the renderer key, then it will call that renderer. If no renderer is found, it just call super
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
This is mainly for calling render and pass parameters like :json, :xml, like the following code,
1 2 3 4 5 6 7 |
|
Since :json is a registered renderer in ActionController::Renderers, it will call that renderer. You can also register your own renderer by calling ActionController::Renderers.add.
If in ActionController::Renderers#render_to_body, it doesn’t find a renderer, then it will call super, which is ActionController::Rendering#render_to_body. Let’s look at what this module does in the method.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
|
Notice that this method calls super first, it only call _render_in_priorities if super returns nothing.
In _render_in_priorities it searches the RENDER_FORMATS_IN_PRIORITY one by one, and return the option value if it finds the format.
In this module when it calls super, it is calling ActionView::Rendering#render_to_body method. Let’s have a look.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
It turns out that here is the the meat we are looking for. The render_to_body calls _render_template, and for the _render_template, it calls view_renderer.render(view_context, options).
The view_renderer is an instance of ActionView::Renderer, and when it’s initialized, it’s passing a lookup_context object, which is an instance of ActionView::LookupContext. The ActionView::LookupContext contains all information about looking for a template based on the options. So in next part we will check this class in detail, and check how LookupContext, ViewPaths, PathSet work together to find the template.
]]>In this first part we firstly check how the render works. Notice that we check Rails 4.2 source code. If you look at another version, the implementation may be slightly different.
The entry point for the render is from the AbstractController::Rendering#render method. The AbstractController is a module shared by ActionController and ActionMailer. Since these two modules share a lot of functionalities, Rails extract those same functionalities into AbstractController. Let’s have a look at this method.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
In our controller, we could call render method directly. For example, we can call render ‘new’ to render the new.html.erb template. Or if we don’t call render explicitly, there is a module ActionController::ImplicitRender which will call a default render.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
The send_action will be called when the action of a controller is triggered. It first calls super, then if in the action it doesn’t render anything, the performed? will return false. So default_render is called. We can see that when call default_render it just calls render without any arguments.
In AbstractController::Rendering#render method, it firstly calls _normalize_render then calls render_to_body. The _normalize_render returns an options object which is a Hash. In this part we will examine the _normalize_render method to see how the options is generated.
Let’s see how _normalize_render is implemented.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
We see that it calls _normalize_args and _normalize_options methods. The _normalize_args and _normalize_options have different purposes.
The _normalize_args is to convert all args into an options hash. For example when we call render method, we could call like this,
1
|
|
Here the first argument ‘new’ is a String, and _normalize_args is responsible to put this first argument in options hash and give it an appropriate key.
Let’s see how it’s implemented.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
We see that this method by default almost does nothing, if the action is a Hash, it returns action, otherwise if action is for example, a string, it returns the second parameter which is the options hash.
Notice that ApplicationController includes some other modules which override this method, as we will see later.
The _normalize_options method is for the modules to include other options. For a Rails application, the ApplicationController extends from ActionController::Base, and ActionController::Base includes a lot of modules, each module could override this method and add some other options.
Let’s firstly check how this method is implemented in AbstractController::Rendering.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
By default this method does nothing. But it will be overridden in other modules.
Rails source code is complex, one reason is because there are many modules could override other modules’s methods. For example, in a ArticlesController, let’s see its ancestors,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
|
We can see that for a typical controller, it has a lot of ancestors and most of them are modules. All modules after AbstractController::Rendering could override its methods. I have created a gist to check which ancestors implement a method.
1 2 3 4 5 6 7 |
|
Run the above code in a rails console, then you can call ClassName.ancestors_that_implement_instance_method to check what ancestors implement a method.
Let’s first see what ancestors override the _normalize_args method,
1 2 3 4 |
|
Two modules override this instance method: ActionView::Rendering and ActionController::Rendering. Let’s look them in order from top to down.
Let’s look at ActionView::Rendering first,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
We can see that for the first argument action, if it’s a string and the string include ‘/’, the key for this argument is :file, and if it doesn’t include ‘/’, the key is :action.
So if we call render ‘new’, the options will be { action: ‘new’ }, if we call render ‘articles/new’, the options will be { file: ‘articles/new’ }
Now let’s see how ActionController::Rendering overrides this method
1 2 3 4 5 6 7 8 9 10 11 |
|
We can see that for this override, if the method is passed a block, it will set the block to options[:update]
Just like _normalize_args, let’s examine what modules override _normalize_options. We can see following modules implement _normalize_options: [ActionController::Rendering, ActionView::Layouts, ActionView::Rendering, AbstractController::Rendering].
Let’s check ActionView::Rendering first,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
We can see that it addes three options by default,
So what’s in options[:prefixes], let’s see how _prefixes method is implemented.
The AbstractController::Rendering module includes ActionView::ViewPaths module. And _prefixes method is implemented there.
ActionView::ViewPaths is an important module which we will examine in more details in later parts. It’s to manage the view paths for controllers. For example, by default Rails will append a view path #{Rails.root}app/views so the application knows to search templates in that specific view path.
For now let’s just focus on _prefixes method.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
The _prefixes just calls the class method _prefixes.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
Notice that Rails 4.2 implements that method and handles some deprecated parent prefixes. In above code I omitted that handling for clarity.
This ActionView::ViewPaths._prefixes calls recursively. it append local_prefixes with superclass’s _prefixes. The local_prefixes has just one element: controller_path.
The controller_path method is very simple, it’s implemented in AbstractController::Base. For example, for a controller ArticlesController, its controller_path will be articles, and for a controller Articles::CommentsController, its controller_path will be articles/comments
So the _prefixes method first gets it’s parent prefixes, and then prepends the current controller_path to the front.
For example, if our application has a ArticlesController, in our rails controller, we can call the following code to show the _prefixes.
1 2 3 4 |
|
We see that the prefixes contains two elements: articles and application. It’s because the ApplicationController extends from ActionController::Base, but ActionController::Base is abstract.
So now we can see that for ArticlesController#index action, when we just call render without any arguments, the options will contains following elements,
Now let’s see next one, how ActionView::Layouts overrides _normalize_options method,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
We can see that if the options[:layout] is not set, the default layout is :default. And then options[:layout] is set to the result returned by _layout_for_option.
If you are interested you could check how _layout_for_options is implemented. When this module searches for the layout it will search “app/views/layouts/#{class_name.underscore}.rb” first, if it doesn’t found, then it will search super class. Since when a rails application is generated, an application.html.erb will be put in app/views/layouts and since by default all controllers’ parent is ApplicationController, so by default this layout will be used.
Finally let’s see how ActionController::Rendering overrides _normalize_options
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
So this method just process :html, :nothing and :status options, which is straight forward.
So finally let’s see when we call render in ArticlesController#index without any arguments, the options will contains following values,
Now we know how the options is normalized. And when Rails determines which template to render, it will extract details from the options. And we will look at how Rails determines template in later parts.
]]>Capistrano is a remote server automation and deployment tool written in Ruby. Capistrano 3 extends Rake DSL with its own set of DSL. I recommend you read the documents on Capistrano website.
In Gemfile we add the following gems,
1 2 3 4 5 6 7 8 9 10 |
|
The capistrano-rails, capistrano-bundler, and capistrano-rbenv are capistrano plugins. Let’s focus on capistrano first.
After running bundle install, if we run bundle exec cap install, it will generate following folders and files, actually when we copy the files from the capistrano template in part 2, it does the same thing.
1 2 3 4 5 6 7 8 9 |
|
To understand Capistrano, the best way is to read its source code. Actually it’s not that hard as the source code of Capistrano is pretty easy to understand.
Capistrano defines a DSL. For those interested, can check the capistrano source code in folder lib/capistrano/dsl
You can set some attributes by using set, and then later use fetch to get the attribute value by key. In our sample application config/deploy.rb, you can see it sets a lot of attributes.
The following code snippets illustrate its use.
1 2 3 4 5 6 7 8 |
|
Think about our deployment, at least we need a database server, an application server which is unicorn, and a web server which is Nginx or Apache. Sometimes db, app and web are on the same server, sometimes they are installed on different servers. Capistrano defines these as roles. For example, in our sample application, we define one server which has three roles,
1
|
|
So later if we want to want to do something on the server that acts as web and app role, we could use roles or release_roles method,
For example, the following code will create directory on the server that act as web and app role,
1 2 3 4 5 |
|
If the web and app role point to two different servers, each server will create the directory. If we use :all, it will get all servers.
Capistrano 3 actually is an extension of Rake tasks. So when we run bundle exec cap production deploy, it will run the deploy Rake task.
Let’s have a look at the definition of the deploy task. It’s in Capistrano source code /lib/capistrano/tasks/framework.rake
1 2 3 4 5 6 7 8 9 10 |
|
So we can see that the deploy task runs the following 8 tasks in sequence, * deploy:starting * deploy:started * deploy:updating * deploy:updated * deploy:publishing * deploy:published * deploy:finishing * deploy:finished
If you are interested you should check the source code of those tasks. For example, in task deploy:updating it will invoke deploy:symlink:shared, which in turns invokes deploy:symlink:link_files, let’s see how this task is implemented
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
|
We can see that it will fetch the linked_files property, for our deploy sample application, we have set it to %w{config/database.yml config/secrets.yml}, so for each file, it will created a symbolic link in release_path, which points to the file in shared_path, where is release_path? Let’s have a look at Capistrano source code /lib/capistrano/dsl/paths.rb
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
So the release_path by default is the current_path, which is the sub folder current under deploy_path, which is set by deploy_to. Since in deploy_sample application, we already set deploy_to to “/product/#{fetch(:full_app_name)}”, so for our case, release_path will be /product/deploy_sample_production/current.
For ssh, Capistrano depends on sshkit, if you are interested, you could check the documents on its website.
In Capistrano we could customize the deploy process by injecting our own tasks. For example, in config/deploy.rb in our sample application, it defines
1
|
|
So after the task deploy:symlink:shared, it will run deploy:compile_assets_locally, which is to run rake assets:precompile locally and then upload the compiled assets to the server. You could use before or after to inject tasks before or after other tasks.
So now you should have an idea how Capistrano works. It’s just a series of Rake tasks which ssh to the server to execute commands to do the deployment. In my opinion the Capistrano source code is quite easy to understand, you could check more details if you like.
]]>In this part we will execute many commands, some will be executed in the host machine (our Laptop or iMac for example), and some will be executed in the Vagrant VM. I will show different prompt when executing. On host machine the prompt will be like host:~/rails_projects/deploy_sample$, and on VM the prompt will be vagrant@vagrant-ubuntu-trusty-64:~$, or if we su as deploy, it will be deploy@vagrant-ubuntu-trusty-64:~$
Just like we use an user postgres to manage our database, we should also use a standalone user to manage our application. We name this user deploy. And now let’s create this user.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
Set a password for deploy user and accept all default options.
Since we already add deploy user to sudo group. It can run sudo commands. However the user needs to input his password when sudo. When we run the capistrano deploy command later, it will run sudo commands. So I want to update this user so that he doesn’t need to type password when running sodo commands. For this we need to update the /etc/sudoers file. It’s recommended to use the application visudo to update it.
1
|
|
Add the following line to this file, and then type Ctrl + O to save it as /etc/sudoers, and type Ctrl + X to exit.
1
|
|
Notice that since we already add deploy user to sudo group, this line should be added under the line %sudo ALL=(ALL:ALL) ALL so it will take effect.
Our deployment organization is like following figure,
Our host communicates with VM by SSH, and also when the VM sync code from github, it also uses SSH. So we need two public/private keypairs here. One will be generated in VM and used to communicate with Github. And One will be generated in host machine and used to communicate with VM during deployment.
Let’s firstly generate a keypair for deploy user in VM. The public key for this keypair will be configured in github so the VM can download code from github.
Let’s firstly su as deploy
1 2 3 |
|
Now we go to the ~/.ssh folder and create a keypair.
1 2 3 4 |
|
The climber2002@gmail.com is my email, and you should use your email here.
Accept default options, and after generation two files should be created, id_rsa and id_rsa.pub.
We can run cat id_rsa.pub and copy its content, and then configure this key in github.
In github, choose Settings –> SSH keys –> Add SSH key, and paste the public key, as the following snapshot.
Now we need to add the key of our host to the Vagrant VM. Let’s see if a key already existed.
1 2 3 |
|
Normally you should already have id_rsa and id_rsa.pub. If not run the ssh-keygen again in host to generate the SSH key.
And then lets copy the content of id_rsa.pub and copy it to VM’s ~/.ssh/authorized_keys. If this file doesn’t exist, create it. And then paste the content in id_rsa.pub in this file as a standalone line.
1
|
|
And also I updated the ssh config file /etc/ssh/sshd_config and changed the PasswordAuthentication yes to PasswordAuthentication no so we disable the password authentication, and only use Publickey authentication which is more secure.
After that let’s restart ssh service
1
|
|
Now from your host server, you should be able to ssh into the Vagrant VM.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
|
Now we install the Rbenv. We will use Rbenv installer to ease our installation. Since we will use deploy user as our application owner. Make sure you firstly su as deploy.
1
|
|
After the installation is complete, add following lines to ~/.bashrc, pay attention that you need to add it at the top of the file. Since the script will return if it’s not running interactively.
1 2 3 4 5 6 |
|
After we run . ~/.bashrc to take the change into effect.
Now we need to install some required packages on Ubuntu.
1
|
|
After the necessary packages are installed, lets install Ruby 2.1.4
1 2 |
|
This will download and compile Ruby 2.1.4 so it will take some time. Let’s grab a cup of coffee~~
After installed ruby 2.1.4 let’s install bundler
1
|
|
For my application, I want to deploy the application in /product folder, so let’s create that folder and change the owner to deploy
1 2 |
|
Now we will try to deploy with Capistrano. We will use the capistrano 3 template to help us. Don’t worry if you don’t fully understand, I will explain it in next part.
Now firstly let’s clone the capistrano template in our host station. For me, both this project and the deploy_sample project are in the same folder which is ~/rails_projects
1
|
|
And then in Gemfile of our deploy_sample project, let’s add the necessary gems.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
After that let’s run bundle install to install all the gems.
The capistrano 3 templates added some customized tasks, so lets copy all the files from the template to our project.
1 2 3 |
|
The Capistrano templates has some template files in config/shared folder, these files will be copied to a shared folder in VM when we run the task deploy:setup_config. Notice that we also want secrets.yml to be in the shared folder, so it won’t be committed in our git repository. So let’s copy this file into shared folder also. We change the filename to secrets.yml.erb as the task will render the erb file to secrets.yml during setup.
1
|
|
Now we need to update the config/deploy.rb file as following,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
|
We changed the following things, 1. We changed the application to deploy_sample, which is our application name 2. We changed repo_url to git@github.com:climber2002/deploy_sample.git, which is our git repository 3. We changed the rbenv_type from :system to :user, and rbenv_ruby to 2.1.4, which is the version we installed just now 4. We changed the linked_files to add secrets.yml, the linked files will be stored in share folder but they will be created a symbolic link in config folder. 5. We changed the config_files to add secrets.yml, since we also want to copy this file to shared folder.
Now let’s update config/deploy/production.rb file as following,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
We changed following things for this file 1. We changed the server to 192.168.22.10, which is our VM IP address. 2. We changed deploy_to to /product/#{fetch(:full_app_name)} since we want to deploy to /product folder.
During real deployment you should also change the server_name to the correct domain name for your server.
Now let’s run the command to setup deploy.
1
|
|
This step will copy all shared folders to /product/deploy_sample_production/shared/config folder, let’s check that folder in our VM,
1 2 3 4 5 6 7 8 9 10 |
|
We can see that some files are copied to the /product/deploy_sample_production/shared/config folder. Now we need to update the database.yml and secrets.yml.
For database.yml, let’s firstly copy the database.example.yml to database.yml
1 2 |
|
We changed the content as following,
1 2 3 4 5 6 7 8 9 10 11 |
|
We set the username and password to the value we created in Part 1, and also we set host to localhost since we installed database and application on same server.
Now we need to update secrets.yml, firstly lets generate a secret from our project folder
1 2 |
|
Now we copy this value to secrets.yml, of course you can also use an environment variable. After update our secrets.yml look like this,
1 2 3 4 5 6 7 8 9 10 |
|
the production:secret_key_base is set to the value we just created.
Now all preparation has been done, now let’s do the deploy!
Let’s run following command,
1
|
|
If everything runs correctly, after a long logs the deployment should be successful. And from your browser you can access http://192.168.22.10/articles to see the Article scaffold we’ve created.
In this part we’ve introduced the steps to deploy by using Capistrano 3. And we utilized a Capistrano template. But you may be wondering what Capistrano has done for us. So in next part I will introduct what has been done in this template.
]]>In this first part I will describe how to install the web server and database. Here I will use Nginx, Unicorn and PostgreSQL.
To demonstrate the deployment, I will create a sample Rails 4.2 application called deploy_sample.
1
|
|
And then I update the .gitignore file to exclude database.yml and secrets.yml from committing to repository.
Add the following lines to .gitignore
1 2 |
|
And then we copy the database.yml and secrets.yml to an example yml so let’s say if your colleagues need to work on this project, they could copy the example yml back and set their configurations.
1 2 |
|
Now let’s init the git repository and commit our project.
1 2 3 |
|
We will use github as our remote repository, so let’s create a repository on github. And then follow its instructions, let’s push the project.
1 2 |
|
After that if we refresh the repository on github, we could see the content that we have committed, like the following snapshot.
Now let’s create some models, for simplicity I will just create a model called Article using scaffold.
1
|
|
We can run the migrations and test the application is ok from accessing *http://localhost:3000*
1 2 3 |
|
After we can commit our changes,
1 2 3 |
|
Now this sample application will be used for our deployment
For demonstration I will use Vagrant to create a VM. But it should apply to any VPS too. Firstly you should install VirtualBox and Vagrant. The installation should be straight forward.
Then we create an empty folder in ~/Sites and init the Vagrant.
1 2 3 |
|
The vagrant init command creates a new file called Vagantfile. This file is configured to use Ubuntu 14.04 LTS server, codenamed “trusty”.
And I want to assign this VM a private IP address, so we can access it from host server later. So open the Vagrantfile and add the following line.
1
|
|
It assigns the IP address “192.168.22.10” to the VM. Later we can see that we can access the nginx by using this IP address.
Now in this folder we could type vagrant up to start the VM. If it’s the first time it will take some time as it needs to download the VM image from the internet.
After the VM is up, we could type vagrant ssh to ssh into the VM. Notice the prompt will change. If you are not sure whether you are in host server or the VM, just check the prompt.
1
|
|
After our VM is up we could install some basic software.
So in the VM, we type following commands,
1 2 |
|
Now let’s install Nginx. As the default Nginx distribution may be out of date, let’s add Nginx repository and install it.
1 2 3 |
|
After installation Nginx should start automatically. We could check its status by executing sudo service nginx status. And we could also access http://192.168.22.10 from our browser.
Now let’s install PostgreSQL. And again we add another repository.
1 2 3 |
|
The libpq-dev library is needed for building the ‘pg’ rubygem.
Installing PostgreSQL will create a user named ‘postgres’. This user is the owner of PostgreSQL.
Let’s create a password for postgres.
1 2 3 4 |
|
In PostgreSQL the user is represented by a Role. Now let’s create a role named deploy_sample.
1 2 3 4 5 6 7 |
|
The above lines will create a user deploy_sample and a database deploy_sample_production and set deploy_sample as its owner. The last command \q is to exit psql.
In next part we will use capistrano template for our deployment. And in that template it will also deploy Monit. According to its official website, “Monit is a small Open Source utility for managing and monitoring Unix systems. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.”
So let’s also install that.
1
|
|
After installation we can run sudo service monit status to confirm that monit service is already started.
We need a Javascript runtime so we are going to install Nodejs.
1
|
|
In this part we setup a VM server and installed Nginx and PostgreSQL. In next part we will see how to deploy our rails app by using capistrano.
]]>Using apt-get to install PostgreSQL is very simple. Just execute the following two commands,
1 2 |
|
Here on my server it will install PostgreSQL 9.3.
During the installation a user named ‘postgres’ will be created automatically and this user is used to start postgresql.
So we should change the password of user ‘postgres’.
1 2 3 4 |
|
The installation will put configuration files in /etc/postgresql/9.3/main by default.
1 2 3 4 5 6 7 8 9 10 |
|
Sometimes to improve performance, we want put postgres data files in its own disk. For example, we may mount a disk in folder /database and we want to put all postgres data file there. So let’s do that.
Suppose the /database folder already exists. We need to change its owner to postgres firstly.
1
|
|
Now we need to initialize this folder as a data folder
1 2 |
|
We must su as postgres first since this user also owns the server process.
Now let’s stop server by running sudo service postgresql stop.
1
|
|
Now we need to update the file /etc/postgresql/9.3/main/postgresql.conf,
change data_directory = ‘/var/lib/postgresql/9.3/main’ to data_directory = ‘/database’
PS: You need to stop the server first before updating the data_directory location. Otherwise the stop will fail and you need to kill the process manually.
After update the configuration file we need to restart postgresql server.
1
|
|
Firstly I use poltergeist Javascript driver, so in my rails_helper.rb it has following configurations,
1 2 3 4 5 6 |
|
When a input box is selectized like the figure above, we can check it generates following DOM structure in our HTML,
The input ‘#properties-input’ is our original input, it’s set ‘display: none’ css property, and Selectize replaces it with another input, and the ‘div.item’ contains the items that we already selected, the ‘div.selectize-dropdown’ is shown as a dropdown when we click the input and lets us to choose another item.
So if we needs to select an item, we could do like this,
1 2 3 4 |
|
Here I use a xpath to find the parent of the input box that we selectized, and then within this parent we click the ‘div.selectize-input’, and the dropdown will show and we run find('div.option', :text => 'Screen Size').click
to click the item that contains the text ‘Screen Size’.
Selectize also provides Ajax support, when you type something in the input, it could send Ajax requests to server to populate the items. So here we simulate the input,
1 2 3 |
|
This is to simulate input a ‘u’ character in the input, and if you configure the load option, it will send a Ajax request to the server to populate the items.
I’ve created a SelectizeHelpers class to simulate some common events,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
|
The selectize_click(id)
is to click the selectize input, the select_option
is to select an option whose text matches the text. And set_text
is to set some text in the input.
For the methods above, the id we should pass the id of the input that we apply the selectize.
]]>Firstly we need to update our OS by running following command,
1
|
|
In some cases on my server it shows the following warnings,
1 2 3 4 5 6 7 |
|
The solution is to edit the /etc/default/locale and add the following contents,
1 2 |
|
Next we need to install the following necessary packages by running,
1
|
|
The python-software-properties allows us to easily manage your distribution and independent software vendor software sources. It will be useful when we install Postgresql and Nginx.
The nodejs is a Javascript runtime for Rails.
Now we need to install a Ruby runtime. I choose rbenv instead of rvm, which is more lightweight for production.
Let’s install rbenv by running the following command,
1
|
|
After installation, following its instructions, add the following content at the top of ~/.bash_profile
1 2 3 4 5 6 |
|
After that run source ~/.bash_profile
to make the changed effective.
Now we install Ruby 2.1.2 and make it global by running following commands,
1 2 3 |
|
The install could take a while so grab a cup of coffee.
After Ruby 2.1.2 is installed, we can install bundler by typing,
1 2 |
|
Now let’s install Nginx. Let’s add the repository of Nginx by typing following command,
1
|
|
Now we need to invoke sudo apt-get update
again since we added a new software source.
1
|
|
Now let’s install Nginx by invoking,
1
|
|
Let’s check Nginx status by typing
1
|
|
If it’s not running, we can start it by running following command,
1
|
|
Now we need to install unicorn if you haven’t done so.
Add gem 'unicorn'
in your Gemfile and run bundle install
.
And after that you can run unicorn_rails
to start it. The default port of unicorn is 8080.
And then create unicorn config file config/unicorn.rb which has following contents,
1 2 3 4 5 6 7 |
|
Here we set worker_processes to 5, and in the server our application will be at /data/xhoppe. And the stdout_path and stderr_path are set to /data/xhoppe/log/unicorn.log.
Now let’s install Postgresql.
I want to install Postgresql 9.3 but the Postgresql version in default repository is 9.1. So we install Postgresql 9.3 by typing following commands, refer here
1 2 3 4 5 |
|
Now let’s create a Postgresql role, open psql by running sudo su -c psql postgres
and type following commands,
1 2 3 |
|
And then update the /etc/nginx/sites-enabled/default file,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
|
The upstream unicorn matches the listen "/tmp/unicorn.sock"
in config/unicorn.rb
After that, we need to run service nginx restart
to restart nginx.
Now we use git clone to clone our project in /data/xhoppe
First we define a /data/xhoppe/env.sh which defines some environments,
1 2 3 4 5 |
|
The XHOPPE_DATABASE_PASSWORD is the production database’s password. And the SECRET_KEY_BASE is for rails session, which is generated by running rake secret
.
After that we can run source env.sh
to set these environment variables.
Now we can run the project by running
unicorn -c config/unicorn.rb -E production -D
Now we can access the website from our browser.
]]>Rails.application.initialize!
to initialize the application. And in this blog we will have a detailed examination what has been done in this method.
Let’s have a look what’s in that method,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
we can see that it just calls the run_initializers
method, this method is actually an instance method in a module called Rails::Initializable. The Rails::Application has following hirarchy,
At the bottom is our Sample::Application class defined in config/application.rb. Its super class is Rails::Application which inherits from Rails::Engine. And Rails::Engine’s parent is Rails::Railtie. The Rails::Railtie includes the module Rails::Initializable. The run_initializers
method is defined in that module. The Railtie is to manage the initialization and configuration of each module. In Rails each module such as Active Record, Active Support has its own Railtie class which inherits from Rails::Railtie. After this blog you should understand the main functionality of the Railtie.
Let’s have a look at the module Rails::Initializable. This module provides a set of methods to manage the order of initialization.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
|
If a class includes this module, it can call the class method initializer
to register a initializer to a class instance variable @initializers
. This method accepts three parameters: the name, an option hash which you can define the order of the initializer, and a block, this block accepts one parameter which is normally our Rails.application
instance.
Let’s check the definition of the Initializer class.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
|
In the initialize
method, it accepts a context object, notice that in Rails::Initializable, the initializer
class method pass context as nil. But later a initializer could call bind
to generate a new initializer with the context bind to an object. So if a class includes Rails::Initializable, its @initializers
class instance variable contains a list of initializer templates. And these initializer templates could be instantiated by bind to a context object. And then pay attention that when a class includes Rails::Initializable module, it also has a initializers
instance method, this method actually gets all initializer templates from itself and its ancestors and pass itself as the context.
Let’s write some classes to verify how Rails::Initializable works. I created a file parent.rb in app/models which has following contents,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
|
This class Parent is a normal class which includes Rails::Initializable. Two initializers are defined: config2 and config1. Although we defined config1 after config2, but we passed before: 'config2'
when initialize config1.
Now let’s open a rails console. The Parent should have a class method which returns all initializers. Let’s print the names of the initializers.
1 2 3 4 |
|
And the initializers in the class act as a template, their instance variable @context is nil. Let’s verify that by printing the @context instance variable, we can see they are nil.
1 2 |
|
Now let’s create an instance of Parent. We can see that the instance also has an instance method called initializers. Let’s print the name,
1 2 3 4 |
|
Just now we said that the initializers in the class are like a template, and for the initializers in the object instance, we can think that the initializers are instantiated by setting the @context instance variable. We can verify that by printing the instance variable of all initializers.
1 2 |
|
We can see that the @context has been set to the parent object.
And if we call the run_initializers method,
1 2 3 4 |
|
Notice two things here, first the config1 is executed before the config2. And secondly notice why we can call the config1 and config2 methods in the initializer block? Because when the initializers are run, the self object in the block is set to the @context instance variable. Since our context object is parent, and config1 and config2 are instance methods of this object, of course we can call it.
Now let’s create a subclass Child1 which inherits from Parent, and in this class it defines another initializer whose name is config_in_child1.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Let’s restart our console and try to create an instance of Child1 and call run_initializers.
1 2 3 4 5 6 7 8 9 |
|
We can see that when an instance run its initializers, it will run the initializers in its ancestors first, from top to bottom. Here the two initializers config1 and config2 are executed first, then it’s config_in_child1 in Child1 class.
Now let’s create another class Child2 also has an initializer, similar as Child1.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
And then let’s create one App class which combines one instance of Child1 and Child2.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
This class has two instance variables @child1 and @child2, which is an instance of Child1 and Child2 separately. This class also includes Rails::Initializable, but it overrides the initializers method to return the intializers of both @child1 and @child2. Now let’s open a new rails console, and create one instance of App and call run_initializers
.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
We can see that when we run app.run_initializers
, it runs all initializers in @child1 and @child2. And also the order is maintained. All config1 initializers are executed before config2 initializers.
The Rails::Application class also overrides the initializers method like the App class above. It maintains a list of Railties and run the Rails.application.initialize!
method is called, it run all Railties’ initializers.
And we could open a rails console to display all initializers and its associated binding context.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
This is a very long list. Actually by default there is 108 initializers. If you are interested you can check the Rails source code to see which initializer does what. For example, the :set_load_path is to set the $LOAD_PATH global. So each engine has a chance to modify the load path.
So after this blog you should have a general idea what Rails has done when it calls Rails.application.initialize!
When we run rails s in a Rails application folder, it will call into the bin/rails script. This script has following contents,
1 2 3 4 5 6 7 8 |
|
Firstly it tries to call the spring script which is an application preloader to speed up development by keeping our application in the background. And then it defines the APP_PATH constant, which points to the <app_root>/config/application, and next it requires the config/boot.rb.
The config/boot.rb has following contents,
1 2 3 |
|
Since Rails uses Bundler to manage the rubygems, it defines a Gemfile which declares all dependencies of the application. The ENV['BUNDLE_GEMFILE']
is set to the path of this Gemfile, and require 'bundler/setup'
is calling Bundler to resolve the load paths of your Gem dependencies.
The rails next requires ‘rails/commands’, and let’s have a look at that file,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
We can see that when we pass s or server to the rails script, it will set command to server and then create an instance of Rails::CommandsTasks and call its run_command!
instance method.
The rails/commands/commands_tasks defines the Rails::CommandsTasks
class, let’s check the important part of this class,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
|
In the run_command!
instance method, it checks that if the COMMAND_WHITELIST includes the command passed, then it will just invoke an instance method whose method name matches the command parameter. So in our case, if we passed server it will invoke the server
method, which we show below,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
|
We can see that it creates a Rails::Server instance and then require our application file which is already defined in APP_PATH.
Let’s first have a look at the config/application.rb
The config/application.rb has following contents,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
Firstly it requires the config/boot.rb, since above this file is already required by rails script, the require has no effect here.
Next it requires the ‘rails/all’, this file has following contents, it just load the railtie of each module. Each module has its own railtie, which is to manage the initialization and configuration. We will talk about railties in another blog so I’ll not explain here.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
And then next it requires the gems we defined in Gemfile by calling Bundler.require
. And next our application class is defined. Since I named my application as sample, the class is in Sample
Module.
The Sample::Application extends from Rails::Application. After this class is defined the Rails.application
will be set to an instance of Sample::Application. Why? Let’s have a look at Rails
module.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
We can see that it defines two instance variables in Rails’s singleton class, @application and @app_class. And after the app_class is defined, the Rails.application returns the @application if it’s already defined, if not, it will call app_class.instance to create an instance.
So when does the Rails.app_class is defined? Let’s have a look at the Rails::Application class.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
We can see that if there is a class inherits it, it will set the Rails.app_class to the inherited class, in our case Sample::Application. So after class Sample::Application is defined, the Rails.app_class is set, and then if we call Rails.application
, the application instance will be initialized.
Notice that the new
method in Rails::Application is set to private, so we can only call instance
to generate an instance. And this instance is a singleton, which is reasonable since one Rails application should have only one application instance.
After the application instance is instantiated, we go back to Rails::CommandsTasks#server method, it will call Dir.chdir(Rails.application.root)
to go to application root folder, and then call server.start
. Let’s have a look how the Rails::Server is defined.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
|
We can see that Rails::Server extends from ::Rack::Server. Rack is a specification which defines an interface between the web server and application. To support Rack, you need to define an app object, it has a call
method, which takes the environment hash as a parameter, and returning an Array with three elements:
Although this interface is very simple, you can configure middleware around the app and intercept the request and response.
Rack is also a rubygem. It provides a set of middleware such as logging, set content length in HTTP response, etc. And it also defines a simple DSL to configure the middleware and app, the config file has ru extension and Rack will load a file called config.ru by default.
Let’s have a look at the ::Rack::Server class,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
|
I only leave the relevant code here and omitted some parts such as option parsing. We can see that the start
method creates a @_server
instance variable which is a handler represents the web server interface, it has a run
method which accepts the app. The app is the Rack app object we mentioned above, it’s an object which reponds to call
method which accepts an environment hash and returns an array with 3 elements: the HTTP status code, response HTTP headers and the body.
The wrapped_app
calls the build_app
method which just wraps some default middleware around the app. And the app
method actually sets the @app
instance variable, which is actually read from the config.ru file (by build_app_from_string
method).
Every rails application has a config.ru file at root folder. Let’s have a look at that file,
1 2 3 4 |
|
This file is actually a ruby file, it first requires the config/environment.rb file and then calls run Rails.application
. The run
method is defined in Rack DSL, and it accepts the app object. So we can imagine that if we call Rails.application.respond_to?(:call)
it should return true.
Let’s have a look what’s inside config/environment.rb.
1 2 3 4 5 |
|
It just requires config/application.rb (which we already required) and then initialize the application by calling Rails.application.initialize!
. After that the initialization process is finished and the Rails.application
could be used as Rack application.
In this part we had a detailed look at the Rails initialization process and we will dig into Railties and Rails.application.initialize!
in following series.
My folder structure is like following figure,
According to the Rails documents, I should create a file index.js and index.css under #{Rails.root}/vendor/assets/tshop, but that doesn’t work for me. Actually I named the files as tshop.js and tshop.css.scss, as the figure above. So in the application.css and application.js I chould load the tshop library by calling,
1 2 3 4 5 6 |
|
and
1
|
|
And in the tshop.css.scss, I could load the file #{Rails.root}/vendor/assets/tshop/stylesheets/style.css by using following code,
1 2 3 4 5 6 |
|
And in tshop.js, the file #{Rails.root}/vendor/assets/tshop/javascripts/script.js could be loaded by following code,
1
|
|
In the templates, I need to change the background image url, for example, in style.css,
1 2 3 4 |
|
This image file will be in #{Rails.root}/vendor/assets/tshop/stylesheets/parallax/people-collage.jpg.
Here are the organization I used for my Rails 4 application. The point is to use library.css and library.js to name the index files instead of index.css and index.js.
]]>The first thing we need to do is to create a Province model. Like how we created the Country model in part 1, we create a Province model, and then import the data in shape file into our PostGIS database. The main part in the migration is as following,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
|
The province data is in shape file GAB_adm1.shp, and we fetch the province name and geometry from the shape file and import them into provinces table (In the shape file province name is stored as attribute name_1).
Now we need to fetch data from database and encode them as GeoJSON. GeoJSON is a JSON format for encoding geographic data structures. The following example is stolen from the GeoJSON website. We can see that in addition to the geometry data, the JSON can also encode additional properties.
1 2 3 4 5 6 7 8 9 10 |
|
We will mainly use the rgeo-geojson gem to do the encoding. In rgeo-geojson gem, it converts a geometry type to a feature and then encode the feature to a hash, and then the hash can be rendered as a JSON.
1 2 3 4 5 6 |
|
We create a Concern named Featurable, which will add a featurable class method into the class which includes it. So for example in Province class, we include the Featurable like following,
1 2 3 4 5 |
|
The featurable accepts two parameters, the first is mandatory, which is the name of the attribute which contains the geometry data, the second parameter is an optional array of attribute names, when encoding it as GeoJSON, those attributes will be encoded as attributes. Here we encode the name attribute as the property of the feature. Now the Province class will have a instance method named to_feature, which returns a RGeo::GeoJSON::Feature.
1 2 3 |
|
The result will display as following, it includes the name as its property and also has an id.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Now let’s have a look at how Featurable is implemented.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
In the implementation of featurable class method, when this method is called, the class will call define_method to define an instance method called to_feature and from line 9 to 13 it will generates a hash named properties whose key is the property name and value is the property value. And then on line 14 it calls the RGeo::GeoJSON::EntityFactory#feature method to create the feature and return it.
On line 21 it defines another class method called to_feature_collection, this is to convert a collection of models to a feature collection, for example the following code shows how to encode all 9 provinces as a feature collection.
1 2 |
|
Now let’s create a ProvincesController to render all 9 provinces as GeoJSON.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
We can see that the index method responds to two formats, in the json format it will call Province.to_feature_collection to create the feature collection, and then calls RGeo::GeoJSON.encode(feature_collection) to encode the feature as a hash, and last calls render to render the hash as a JSON string.
In the views/provinces/index.html.erb, the main part is as following,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
From line 2 to line 4 it creates an info box at the RIGHT TOP to display province name. And from line 6 to line 10 it adds a mouseover event listener, so when the mouse is over the province region, it set the text of the info box to the name property of the feature. And on line 12 it defines a mouseout event listener so the style is reverted when the mouse is out of the province region.
So in this part we showed how to fetch geometry data from PostGIS database and render them on google map. The source code for this part is at github.
]]>To work on PostGIS and Google Maps of course you need to learn them. This series is not a complete tutorial. To study PostGIS, the definite reference is PostGIS in Action by Regina O. Obe and Leo S. Hsu. Currently the second edition is in MEAP state and you can buy the ebook at manning website. To study Google maps, the documentation at google website is already awesome, but if you prefer a book form, the book Google Maps JavaScript API Cookbook by Alper Dincer and Balkan Uraz is a good choice, which you can buy the kindle version at amazon
In the project to integrate PostGIS with Rails, I utilized some rubygems from RGeo. The original author of those rubygems, Daniel Azuma, has created a series of articles on his website, which is very helpful. Actually this series has been inspired from his works.
In this first blog I’ll describe how to create a rails project and how to create migrations to import GIS data from shape files to our database.
To work on PostGIS of course you need to install it first. If you work on a Mac OS environment like me, using the PostgreSQL App is the easiest way. This package already includes PostGIS 2.1, so you just need to enable it, which we will introduce later.
And since we use the RGeo and some of its addons, you need to install GEOS and Proj, which RGeo depends on. For Mac users the easiest way is to install the frameworks here
Now let’s create a rails application and use PostgreSQL database (I use Rails 4.1).
1
|
|
Here my application name is schoolpro. Now open the Gemfile and add the activerecord-postgis-adapter gem.
1
|
|
And then in the database.yml, let’s change the adapter from postgresql to postgis
1
|
|
Now let’s create the database by running rake db:create
1
|
|
After the database is created, we need to enable the PostGIS extension by running following command
1
|
|
In this application I need to import some data from shape files to the database. The shapefile is a flat file format for geospatial data originally developed by ESRI for storing sets of geographic features. It supports certain vector shapes— points, lines, and polygons— along with associated attributes. Although shapefile began as a proprietary format, the format specification is readily available, and it is now a de facto standard for large datasets. A shapefile actually consists of three (and sometimes more) related files, each with the same base filename but different extensions. The main file has the extension “.shp” and contains the geometric data itself in a binary format. An auxiliary “.shx” file provides a simple flat index allowing random access into the shapefile. A second auxiliary “.dbf” file provides the attribute data in dBASE format. All shapefiles should have those three core files, although some shapefiles may include additional files containing coordinate system, spatial index, or other related information.
The shape files I got is from gdam, which is a spatial database of the locations of the world’s administration areas. What I need is the shape files of Gabon. Just go to the download page, select Gabon and File format as Shapefile and download it. The downloaded is a zip file, unzip it you will get 3 sets of shapefiles:
Let’s create a folder shpfiles under #{Rails.root}/db and copy all these files there. These files will be used in our migrations.
My goal is to import the country polygon data from a shape file into the database in rails migrations. Now let’s import the countries shapefile. Firstly we need to create a Country model. So let’s create a model Country and add following content in the migration file,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
Here we specified 3 attributes, name is country name, the iso_code is 3 letter code, for Gabon it’s GAB. The t.multi_polygon is to create a multipolygon geometry column named geom, which represents that this column can store multipolygon type in PostGIS. And we set the SRID to 4326, which is most common in shape files. Although when Google maps displays its maps, it uses SRID 3857 however when draw vectors on it it still expects the vector data to be SRID to 4326. For a more explanation of SRID, consult Daniel’s article
Now let’s import the data from the shape file into our new created countries table. We will use the shp2pgsql tool, which is already included in Postgres App. I assume that you already add the bin path into your PATH environment variable, if not, you can add them in your ~/.bash_profile
1
|
|
Now you can open a new terminal and run shp2pgsql to check its usage,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
|
Now let’s create another migration to import the shapefile,
1
|
|
And in the migration file we have following contents,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
In the up method of the migration, we first run the shp2pgsql in shell command by using Backticks(`) (check this nice blog). The output of shp2pgsql is the SQL scripts and it will be returned from the result of the shell command, and then we store it in variable from_shp_sql. We pass following parameters to the shp2pgsql:
And then at last we pass the path of the shape file and name the generated table name as countries_ref.
And then we run this SQL in a transation to create the countries_ref table and import the data. And then run the following SQL to copy the data from table countries_ref to countries.
1 2 3 |
|
In the countries_ref table we already named the geometry column as geom. The name_engli and iso are attributes from the shape file and when shp2pgsql generate the table, it will name the column name the same as attribute name. At last since we already done the import we call drop_table countries_ref to delete the countries_ref* table.
In the down method, we just delete all data in the countries table.
Generate GeoJSON
GeoJSON is a format for encoding a variety of geographic data structures. Google maps already has native support for it. In this blog let’s create a static geojson file and render it in google map. In later blogs we will generate geojson format from database directly.
RGeo has a nice addon rgeo-geojson which provides GeoJSON encoding and decoding services. Let’s add it in Gemfile:
1 2 |
|
After bundle install, we can open the rails console (I have installed pry-rails to use pry instead of irb),
1 2 3 4 5 6 7 8 |
|
Firstly we get the gabon model by calling Country.first since there is only one row in countries table. For rgeo-geojson, it uses a factory to generate the feature, so we get a factory and then calls the feature method and passes gabon.geom. This geom property is a RGeo geometry instance, and it’s wrapped in feature. Then we call the RGeo::GeoJson.encode feature to encode it to a hash object, then at last we write the content of this hash to a file named gabon.json.
Now let’s move this gabon.json to the public folder so the browser can access it directly.
When we start the server and access http://localhost:3000/gabon.json we should be able to see the content of this file in browser.
Render in google maps
Now let’s render this GeoJSON file on google maps. I have created a MapsController for that, and in the view it will create a google.maps.Map instance and render it. I won’t explain all and you can check the source code in app/views/maps/index.html.erb. (Yeah I know it’s very bad behavior to embed javascript in erb files directly, let’s refactor it later.)
The most important part is to call *map.data.loadGeoJson to load GeoJSON data,
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Now when we access http://localhost:3000/maps we can see that the the gabon area is filled with black color, as following figure,
The repository of this project is at https://github.com/climber2002/schoolpro . The source code of each blog has its own tag so it’s easy to get it. For this part the source code is at https://github.com/climber2002/schoolpro/tree/part1 .
In this part we load the GeoJSON in a static file. In next part we will see how to generate the GeoJSON dynamically.
]]>Download the Postgres App here, which is the quickest way to install PostgreSQL on Mac OS. My version is the latest version Postgres93.
Install XCode, and in the menu XCode –> ‘Open Developer Tool’ –> ‘More Developer Tool’, Choose the ‘Command Line Tools(OS X Mavericks) for Xcode’, download the dmg file and install it.
Install the pg gem with the following command,
1
|
|
Notice the —with-pg-config params, otherwise you may get the following errors,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
|
Normally I want to create a specific role in the database and don’t like the anonymous access. So in my database.yml, I have the following snippets,
1 2 3 4 5 6 7 8 |
|
So here for this project I have a user ebilling whose password is also ebilling, so I created the ebilling user with the following command,
1
|
|
And then open the psql console, and assign the user as superuser and password
1
|
|
Now after bundle install, you can create the database with the rake command,
1
|
|