Testing a Wagtail site with Factory Boy

In a previous tutorial we tested an authentication app. The setup of the test data merely consisted of defining a couple of users. A more complicated app, such as a Wagtail site, requires more data to be set up: users, pages, a menu, other items that we want to test. Some of these elements are dependent on each other. Defining all of the test instances explicitly, as we have done with the authentication app, is possible but is not very flexible. Another strategy is to use fixtures: data from a previous or demo site that is dumped in a file and loaded again before testing. A disadvantage might be that the fixtures file is large and not easy to comprehend or maintain, or that it might take some time to set up a good fixtures file.

An alternative is to use Factory Boy, a tool to generate test data. Factory Boy sets up 'factories' for model instances, in which certain properties, relationships or sequences can be predefined, such as parent-child relationships, generating new instances when calling the factory multiple times, defining default values. Once the factories are set up, setting up the test data becomes considerably easier and much more flexible. Another option would have been to use Wagtail Factories, which has a number of out-of-the-box options.

Start with installing Factory Boy and putting it in requirements.txt:

pip3 install factory_boy

Since we anticipate having quite a number of tests, let's create a directory /tests within our app, where we will put all our test files. We need to create a __init__.py file to make the file discoverable for Django's test runner. Now create a file factories.py where we will put all our factories.

Factory Boy has a special class for Django models. Creating a factory for a model is done as follows:

import factory
from ..models import Theme

class ThemeFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = Theme

Creating an instance of Theme is as simple as:

theme1 = ThemeFactory()

Our model Theme has only one field, called name. We can instruct Factory Boy to automatically generate a new name every time a new Theme instance is created, by using sequences. For the above class ThemeFactory define a field name:

name = factory.Sequence(lambda n: 'Theme number {0}'.format(n))

The first instance that will be created will have a name Theme number 0, the second Theme number 1, etc. We use the Python str.format() function instead of printf style string formatting.

For simple models this works fine. However, Wagtail pages have parent-child relationships, so in order to create something like a PageFactory, we need a way to automatically establish this. This can be done by overriding the _create method. Our strategy is to add an extra argument with key parent, that will point to the parent page of the page we want to create. Our factory is then:

class PageFactory(factory.django.DjangoModelFactory):
    class Meta:
        abstract = True

    # override the _create method, to establish parent-child relationship between pages
    @classmethod
    def _create(cls, model_class, *args, **kwargs):

        try:
            parent = kwargs.pop('parent')
        except KeyError:
            # no parent, appending page to root
            parent = Page.get_first_root_node()

        page = model_class(*args, **kwargs)
        parent.add_child(instance=page)

        return page

We set abstract = True to indicate that this factory is abstract. In the _create method we look for a parent in the keyword arguments, and if it is there, then we add the to-be-created page as a child to this parent. If there is no parent argument, then we append the to-be-created page to the root. To find the root, we use the method get_first_root_node, available through an ancestor of a Wagtail page.

We create a factory for homepages by subclassing from PageFactory:

from ..models import HomePage

class HomePageFactory(PageFactory):
    class Meta:
        model = HomePage

    title = factory.Sequence(lambda n: 'Home page {0}'.format(n))

and the exact same code can be used to create other Page classes, in our case a ThemePageFactory, ThemeIndexPageFactory, ArticleIndexPageFactory (just replace Home with Theme etc.). For an ArticlePageFactory we need a two more things. First we will make sure that article pages have different first_published_at fields, because we intend to have a test that the ordering is done right. This is again achieved by using a sequence:

from datetime import timedelta
from django.utils import timezone

first_published_at = factory.Sequence(lambda n: timezone.now() - timedelta(days=365-n))

Secondly, our ArticlePage model has a many-to-many relationship with the Theme model. This can be established by using a post generation hook, in the following way:

@factory.post_generation
def themes(self, create, extracted, **kwargs):
    if not create:
        # Simple build, do nothing.
        return

    if extracted:
        for theme in extracted:
            self.themes.add(theme)

This allows us to add a tuple of themes to the to-be-created page as relations to the specific page.

Those are all the factories we need to start writing the tests. In the directory /tests create a file test_pages.py. We define our initial data using setUpTestData, just as we did in the previous tutorial. The first two objects to create are a request factory and a site:

from django.test import TestCase, RequestFactory
from wagtail.core.models import Page, Site

class TestPageModels(TestCase):

    @classmethod
    def setUpTestData(cls):
        cls.factory = RequestFactory()
        # Wagtail needs a site, e.g. to define url
        # Page.get_first_root_node() is used by Wagtail to find the root page
        cls.site = Site.objects.create(is_default_site=True, root_page=Page.get_first_root_node())

We will need the request factory in our tests. Wagtail also needs a Site instance, e.g. when constructing the url of a page (see Page and Site model). Now we are ready to construct page instances using our factories. We create a homepage, a themeindexpage and an articleindexpage (both children of the homepage), two themes and two article pages as children of the articleindexpage. We also define some many-to-many relations and set a few fields.

cls.homepage = HomePageFactory()
cls.articleindexpage = ArticleIndexPageFactory(parent=cls.homepage)
cls.themeindexpage = ThemeIndexPageFactory(parent=cls.homepage)
cls.theme1 = ThemeFactory()
cls.theme2 = ThemeFactory()
cls.themepage1 = ThemePageFactory(theme=cls.theme1, parent=cls.themeindexpage)
cls.themepage2 = ThemePageFactory(theme=cls.theme2, parent=cls.themeindexpage)
cls.articlepage1 = ArticlePageFactory(parent=cls.articleindexpage, themes=(cls.theme1, cls.theme2,), featured=True)
cls.articlepage2 = ArticlePageFactory(parent=cls.articleindexpage, themes=(cls.theme1,))

As you can see, now that we have defined our factories, defining instances takes just one line. If we would need more instances for more tests, this can easily be done. Let's write our first test: check whether the articles on a theme page are in reverse chronological order:

def test_articles_on_themepage_are_in_antichronological_order(self):
    articlepageslist = self.themepage1.articlepages()
    publisheddatelist = [articlepage.first_published_at for articlepage in articlepageslist]
    self.assertTrue(publisheddatelist == sorted(publisheddatelist, reverse=True))

We use the method articlepages() of a theme page, extract the dates and check whether they are in reverse chronological order. We run the test with:

python3 manage.py test cms.tests.test_pages

The same test for articles on the articleindexpage has almost the same code:

def test_articles_on_articleindexpage_are_in_antichronological_order(self):
    articlepageslist = self.articleindexpage.articlepages()
    publisheddatelist = [articlepage.first_published_at for articlepage in articlepageslist]
    self.assertTrue(publisheddatelist == sorted(publisheddatelist, reverse=True))

Some more tests: the working of the featured field, the themepages() method and the get_absolute_url() method of an article page:

def test_featured_articles_on_articleindexpage_get_set(self):
    self.assertEqual(len(self.articleindexpage.featured_articlepages()), 1)

def test_themepages_of_articlepage(self):
    themepageslist = self.articlepage1.themepages()
    self.assertEqual(len(themepageslist), 2)
    self.assertEqual(themepageslist[0].url, self.themepage1.url)

def test_get_absolute_url_of_article_page(self):
    self.assertEqual(self.articlepage1.get_absolute_url(), self.articlepage1.url)

Our final test is to test the serve() method of the article page. For that we need a request, which we will create with the request factory instance we defined. The serve() method will turn that request into a TemplateResponse object, but because the RequestFactory does not support middleware not all functionality of the response is available; the status_code field however is. Following the example in the Django docs:

def test_article_page_is_served(self):
    # first create get request to page (serve method needs it)
    request = self.factory.get(self.articlepage1.url)
    # page serve method will return TemplateResponse object (but with only HttpResponse attributes, due to limitations of RequestFactory)
    response = self.articlepage1.serve(request)
    # this will have status code 200, if the page exists
    self.assertEqual(response.status_code, 200)

Now let's see how much of our code is already covered by the preceding tests. As in our previous tutorial, we use Coverage for that:

coverage run --source=cms manage.py test cms
coverage html

Opening index.html in our browser shows that all page related classes in our models are pretty well covered. Time to move on to testing menus and languages.