Robot Has No Heart

Xavier Shay blogs here

A robot that does not have a heart

How I Test Rails Applications

The Rails conventions for testing provide three categories for your tests:

  • Unit. What you write to test your models.
  • Integration. Used to test the interaction among any number of controllers.
  • Functional. Testing the various actions of a single controller.

This tells you where to put your tests, but the type of testing you perform on each part of the system is the same: load fixtures into the database to get the app into the required state, run some part of the system either directly (models) or using provided harnesses (controllers), then verify the expected output.

This techinque is simple, but is only one of a number of ways of testing. As your application grows, you will need to add other approaches to your toolbelt to enable your test suite to continue providing valuable feedback not just on the correctness of your code, but its design as well.

I use a different set of categories for my tests (taken from the GOOS book):

  • Unit. Do our objects do the right thing, and are they convenient to work with?
  • Integration. Does our code work against code we can’t change?
  • Acceptance. Does the whole system work?

Note that these definitions of unit and integration are radically different to how Rails defines them. That is unfortunate, but these definitions are more commonly accepted across other languages and frameworks and I prefer to use them since it facilitates an exchange of information across them. All of the typical Rails tests fall under the “integration” label, leaving two new levels of testing to talk about: unit and acceptance.

Unit Tests

“A test is not a unit test if it talks to the database, communicates across a network, or touches the file system.” – Working with Legacy Code, p. 14

This type of test is typically referred to in the Rails community as a “fast unit test”, which is unfortunate since speed is far from the primary benefit. The primary benefit of unit testing is the feedback it provides on the dependencies in your design. “Design unit tests” would be a better label.

This feedback is absolutely critical in any non-trivial application. Unchecked dependency is crippling, and Rails encourages you not to think about it (most obviously by implicitly autoloading everything).

By unit testing a class you are forced to think about how it interacts with other classes, which leads to simpler dependency trees and simpler programs.

Unit tests tend to (though don’t always have to) make use of mocking to verify interactions between classes. Using rspec-fire is absolutely critical when doing this. It verifies your mocks represent actual objects with no extra effort required in your tests, bridging the gap to statically-typed mocks in languages like Java.

As a guideline, a single unit test shouldn’t take more than 1ms to run.

Acceptance Tests

A Rails integration test doesn’t exercise the entire system, since it uses a harness and doesn’t use the system from the perspective of a user. As one example, you need to post form parameters directly rather than actually filling out the form, making the test both brittle in that if you change your HTML form the test will still pass, and incomplete in that it doesn’t actually load the page up in a browser and verify that Javascript and CSS are not intefering with the submission of the form.

Full system testing was popularized by the cucumber library, but cucumber adds a level of indirection that isn’t useful for most applications. Unless you are actually collaborating with non-technical stakeholders, the extra complexity just gets in your way. RSpec can easily be written in a BDD style without extra libraries.

Theoretically you should only be interacting with the system as a black box, which means no creating fixture data or otherwise messing with the internals of the system in order to set it up correctly. In practice, this tends to be unweildy but I still maintain a strict abstraction so that tests read like black box tests, hiding any internal modification behind an interface that could be implemented by black box interactions, but is “optimized” to use internal knowledge. I’ve had success with the builder pattern, also presented in the GOOS book, but that’s another blog post (i.e. build_registration.with_hosting_request.create).

A common anti-pattern is to try and use transactional fixtures in acceptance tests. Don’t do this. It isn’t executing the full system (so can’t test transaction level functionality) and is prone to flakiness.

An acceptance test will typically take seconds to run, and should only be used for happy-path verification of behaviour. It makes sure that all the pieces hang together correctly. Edge case testing should be done at the unit or integration level. Ideally each new feature should have only one or two acceptance tests.

File Organisation.

I use spec/{unit,integration,acceptance} folders as the parent of all specs. Each type of spec has it’s own helper require, so unit specs require unit_helper rather than spec_helper. Each of those helpers will then require other helpers as appropriate, for instance my rails_helper looks like this (note the hack required to support this layout):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
ENV["RAILS_ENV"] ||= 'test'
require File.expand_path("../../config/environment", __FILE__)

# By default, rspec/rails tags all specs in spec/integration as request specs,
# which is not what we want. There does not appear to be a way to disable this
# behaviour, so below is a copy of rspec/rails.rb with this default behaviour
# commented out.
require 'rspec/core'

RSpec::configure do |c|
  c.backtrace_clean_patterns << /vendor\//
  c.backtrace_clean_patterns << /lib\/rspec\/rails/
end

require 'rspec/rails/extensions'
require 'rspec/rails/view_rendering'
require 'rspec/rails/adapters'
require 'rspec/rails/matchers'
require 'rspec/rails/fixture_support'
require 'rspec/rails/mocks'
require 'rspec/rails/module_inclusion'
# require 'rspec/rails/example' # Commented this out
require 'rspec/rails/vendor/capybara'
require 'rspec/rails/vendor/webrat'

# Added the below, we still want access to some of the example groups
require 'rspec/rails/example/rails_example_group'
require 'rspec/rails/example/controller_example_group'
require 'rspec/rails/example/helper_example_group'

Controllers specs go in spec/integration/controllers, though I’m trending towards using poniard that allows me to test controllers in isolation (spec/unit/controllers).

Helpers are either unit or integration tested depending on the type of work they are doing. If it is domain level logic it can be unit tested (though I tend to use presenters for this, which are also unit tested), but for helpers that layer on top of Rails provided helpers (like link_to or content_tag) they should be integration tested to verify they are using the library in the correct way.

I have used this approach on a number of Rails applications over the last 1-2 years and found it leads to better and more enjoyable code.

  1. Brian says:

    We've been doing nearly the same thing on our latest app and ran into the same quirks with rspec-rails and its automatic classification of specs in spec/integration. The below snippet let us customize this behavior for our model and controller specs; it is an adaptation of how rspec-rails sets its own defaults (see: lib/rspec/rails/example.rb)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    
    # in spec/support/integration_spec_path_configuration.rb
    
    RSpec::configure do |config|
    
      %w(controller model).each do |type|
        example_group_module_name = "#{type.capitalize}ExampleGroup"
        example_group_module = RSpec::Rails.const_get(example_group_module_name)
        spec_directory_name = type.pluralize
    
        config.include example_group_module, type: type.intern, example_group: {
          file_path: config.escaped_path(
            %W(spec integration #{spec_directory_name})
          )
        }
      end
    
    end
    

  2. q says:

    "the extra complexity just gets in your way"

    I'd suggest the same about RSpec--check out minitest! :)

  3. 21croissants says:

    Thanks for sharing.

    Would you mind sharing your unit_helper.rb ?

    I'm playing with rspec-fire and the README does not provide any example.

  4. Xavier Shay says:

    It really shouldn't have much. Here is one off a current project, includes a bonus `fire_value` helper I'm experimenting with.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    
    $LOAD_PATH.unshift("./app/models")
    $LOAD_PATH.unshift("./app/helpers")
    $LOAD_PATH.unshift("./app/support")
    $LOAD_PATH.unshift("./app/controllers")
    
    require 'rspec/fire'
    require 'ostruct'
    
    require 'coverage_helper'
    
    SimpleCov.command_name 'spec:unit'
    
    RSpec.configure do |c|
      c.include(RSpec::Fire)
    end
    
    class FireValueObject < OpenStruct
      include RSpec::Fire::FireDoublable
    
      def initialize(doubled_class, *args)
        args << {} unless Hash === args.last
    
        @__doubled_class_name = doubled_class
        verify_constant_name if RSpec::Fire.configuration.verify_constant_names?
    
        @__checked_methods = :public_instance_methods
        @__method_finder   = :instance_method
    
        super(*args)
      end
    end
    
    def fire_value(klass, attributes = {})
      FireValueObject.new(klass, attributes)
    end
    

  5. John Bachir says:

    nice post. you should make a rspec-rails-goos gem that supports all these practices :-D

  6. http://id.leventali.com/ says:

    Maybe I'm missing something here but do you stub out ActiveRecord::Base somewhere when unit testing your models?

    In your unit_helper.rb I don't see how ActiveRecord would be loaded.

    Thanks
    Levent

  7. Xavier Shay says:

    Under this model, anything using ActiveRecord would be an integration test since those are boundary objects. It is not loaded in unit tests.

  8. Robert Berger says:

    You state that unit tests are for the model, except that anything involving Active Record is not a unit test. In almost every Rails app I've seen every model class extends ActiveRecord::Base. So what does that leave to be unit tested ?

    Or is rspec-fire used to make the model classes unit testable ?

    This approach seems more workable with persistence frameworks in which every attribute must already be explicitly declared in the code (i .e. DataMapper)

    Declaring the attributes solely for testing creates another way in which the test environment can diverge from "reality"

  9. Xavier Shay says:

    Robert: most Rails applications put too much logic in ActiveRecord descendants. They should only deal with persistence (which should be integration tested), and all business logic should be move out to plain Ruby objects.

  10. polo says:

    http://www.burberry-factory.net/
    http://www.shophandbagsonline.com/
    http://www.official-coachoutlet.com/
    http://www.barbour-factory.com/
    http://www.burberry-outlet2014.com/
    http://www.guccibags.us.com/
    http://www.marcjacobsonsale.com/
    http://www.mcmworldwide.ca/
    http://www.guccishoes-uk.com/
    http://www.kate-spades.com/
    http://www.louisvuittonas.com/
    http://www.lv-guccishoesfactory.com/
    http://www.official-mkoutlets.com/
    http://www.official-pradaoutlet.com/
    http://www.michael-korsusa.net/
    http://www.north-facesoutlet.com/
    http://www.moncler-clearance.com/
    http://www.north-faceclearance.com/
    http://www.clothes-mall.com/
    http://www.polo-outlets.com/
    http://www.ralphlauren.so/
    http://www.ralphlaurentshirts.com/
    http://www.ferragamos.in.net/
    http://www.longchampsoutlet.com/
    http://www.abercrombiee.com/
    http://www.barbour-jacketsoutlet.com/
    http://www.michael--korsonline.com/
    http://www.thenorthface.so/
    http://www.beatsbydreoutlet.net/
    http://www.tommyhilfiger.in.net/
    http://www.ralphslauren.co.uk/
    http://www.michaelkors.so/
    http://www.oakleyssunglassoutlet.com/
    http://www.warm-boots.com/
    http://www.woolrich-clearance.com/
    http://www.nike-jordanshoes.com/
    http://www.monsterbeatsbydres.net/
    http://www.canada-gooser.com/
    http://www.bestcustomsonline.com/
    http://coach.mischristmas.com/
    http://www.coach-blackfriday2014.com/
    http://www.coachccoachoutlet.com/
    http://www.coach-clearance.com/
    http://www.coach-factories.net/
    http://www.coach-factorysoutlet.com/
    http://www.coachlosangeles.com/
    http://www.coachoutletstates.com/
    http://www.coach-pursesoutlets.com/
    http://www.hermes-outletonline.com/
    http://www.misblackfriday.com/
    http://www.mischristmas.com/
    http://www.mmoncler-outlet.com/
    http://www.newoutletonlinemall.com/
    http://www.ralphlaurenepolo.com/
    http://www.zxcoachoutlet.com/
    http://michaelkorsoutlet.mischristmas.com/
    http://mcmbackpack.mischristmas.com/
    http://monsterbeats.mischristmas.com/
    http://northfaceoutlet.mischristmas.com/
    http://mk.misblackfriday.com/
    http://coachoutlet.misblackfriday.com/
    http://coachfactory.misblackfriday.com/
    http://uggaustralia.misblackfriday.com/
    http://coachpurses.misblackfriday.com/
    http://coachusa.misblackfriday.com/
    http://coach.misblackfriday.com/
    http://michaelkorss.misblackfriday.com/
    http://michaelkors.misblackfriday.com/
    http://airmax.misblackfriday.com/
    http://michael-kors.misblackfriday.com/
    https://twitter.com/CoachOutlet2014
    https://www.facebook.com/coachoutletstoreonline
    https://www.facebook.com/ralphlaurenoutletonline

Post a comment


(lesstile enabled - surround code blocks with ---)

A pretty flower Another pretty flower