Matthias Kestenholz: Posts about Python

Composition over inheritance: The case for function-based views

2023-08-11T12:00:00Z

Composition over inheritance: The case for function-based views

A recent conversation with Carlton on Mastodon prompted me to write down some of my thoughts re. function- vs class-based views in Django.

The early days

When I started using Django some time after 0.96 and 1.0 all views were function based. Except when you added a class with a def __call__() method yourself – that was always possible but not really comparable to today’s class-based views.

The introduction of class-based views

Class based views (both generic versions and the base View) were introduced to Django in 2010. Judging from the ticket tracker the main motivation was to avoid adding yet another argument to the generic function-based views (GFBV) which were available in Django back then.

The GFBV’s argument count was impressive. Two examples follow:

def object_detail(request, queryset, object_id=None, slug=None,
    slug_field='slug', template_name=None, template_name_field=None,
    template_loader=loader, extra_context=None,
    context_processors=None, template_object_name='object',
    mimetype=None):
    ...

def archive_month(request, year, month, queryset, date_field,
    month_format='%b', template_name=None, template_loader=loader,
    extra_context=None, allow_empty=False, context_processors=None,
    template_object_name='object', mimetype=None, allow_future=False):
    ...

The GFBVs were immediately deprecated when GCBVs were introduced and later removed in 2012.

Class-based views have to be adapted by calling the View.as_view() method; as_view() returns arguably the thing which is viewed (sorry) as the view by Django, it’s the thing which gets called with a request and is expected to return a response. This thing in turn instantiates the view object once per request; this means that self can be used to save request-specific data such as self.request, self.args but also custom attributes.

The GCBV code is extremely factored and decomposed. The Classy Class-Based Views site mentions that the UpdateView has 10 separate ancestors and its code is spread across three files. But, the view code for instantiating a model object and handling a form really isn’t that complex. Most of the complexity is handled by Django itself, in the request handler and in the django.forms package. So, what’s the reason for all this?

Generic views could be simple

I wish that the existing generic views had better building blocks instead of a big hierarchy of mixins and multiple inheritance which is probably not understood by anyone without checking and re-checking the documentation, the code, or the excellent Classy Class-Based Views. Certainly not by me.

In my ideal world, generic views would be composed of small reusable and composable functions which wuld cover 80% of use cases with 20% of the code. And if not, you could copy the whole code of the view, change or introduce a line or two and leave it at that. And since the functions do one thing (but do that well) you can immediately see what they are doing and why. You’d avoid the Hollywood Principle (Don’t call us, we’ll call you) in your code. Sure, your view is called by Django but you don’t have to introduce more and more layers of indirection.

The internet is full of advice that you should prefer composition over inheritance. Let’s try to outline what generic views could look like if they followed the composition paradigm. Note that the goal isn’t to gain points by showing that the resulting code is shorter. One important goal is maintainability by being easier to understand. Another important goal is showing a better path from a beginner’s use of views to an experts understanding of everything underneath it by bridging the gap using more powerful building blocks which don’t leave all the minutiae to you if the defaults don’t work.

Some repetition caused by copy pasting is fine. Not all identical three lines of code are the same. The Three Strikes And You Refactor rule¹ leads to better and more maintainable code than following an extreme interpretation of the DRY (Don’t Repeat Yourself) principle.

ListView and DetailView

I’m going to profit from Django’s shortcuts module and also from feincms3’s shortcuts module which offers functions for rendering pages for single objects or lists of objects. The render_list and render_detail functions implement the same way of determining the template paths as the generic views use (for example <app_name>/<model_name>_detail.html) and the same way of naming context variables (object and <model_name> for the object, object_list and <model_name>_list for the list) as well as pagination but nothing more.

Here’s a possible minimal implementation of a list and detail object generic view:

# _get_queryset runs ._default_manager.all() on models and returns
# everything else as-is. It's the secret sauce which allows using models,
# managers or querysets with get_object_or_404 and friends.
from django.shortcuts import get_object_or_404, _get_queryset
from feincms3.shortcuts import render_list, render_detail

def object_list(request, *, model, paginate_by=None):
    return render_list(request, _get_queryset(model), paginate_by=paginate_by)

def object_detail(request, *, model, slug, slug_field="slug"):
    object = get_object_or_404(model, **{slug_field: slug})
    return render_detail(request, object)

You want to change the way a single object is retrieved? You could do that easily but not by adding configuration-adjacent values in your URLconf but rather by adding a view yourself:

def article_detail(request, year, slug):
    object = get_object_or_404(Article.objects.published(), year=year, slug=slug)
    return render_detail(request, object)

urlpatterns = [
    ...
    path("articles/<year:int>/<slug:slug>/", article_detail, name=...),
    ...
]

I don’t think that was much harder than a hypothetical alternative:

urlpatterns = [
    ...
    path(
        "articles/<year:int>/<slug:slug>/",
        object_detail,
        {
            "model": Article.objects.published(),
            "object_kwargs": ["year", "slug"],
        },
    ),
    ...
]

And think about the internal implementation of the object_detail view. Viewed one additional feature at a time it may be fine but when adding up everything it would probably be quite gross.

The additional benefit is that it shows beginners the way to intermediate skills – writing views isn’t hard, and shouldn’t be.

Finally, the official way of overriding DetailView.get_object() (I think!) doesn’t look that good compared to the def article_detail() view above:

class ArticleDetailView(generic.DetailView):
    def get_object(self, queryset=None):
        if queryset is None:
            queryset = self.get_queryset()
        return get_object_or_404(queryset, year=self.kwargs["year"], slug=self.kwargs["slug"])

Did you know that get_object() has an optional queryset argument? I certainly didn’t. It seems to be used by the date-based generic views but they also have their own get_object() implementation so who knows, really.

Detail view with additional behavior

def article_detail(request, year, slug):
    object = get_object_or_404(Article.objects.published(), year=year, slug=slug)
    data = (request.POST, request.FILES) if request.method == "POST" else ()
    form = CommentForm(*data)
    if form.is_valid():
        form.instance.article = object
        form.save()
        return HttpResponseRedirect(".#comments")
    return render_detail(request, object, {"comment_form": form})

A counterexample would be to move the endpoint which accepts a comment POST request somewhere else. But then you’d also have to keep the different CommentForm instantiations in sync.

You could also override get_context_data() to add the comment form to the context and override post() to instantiate check the form’s validity. But then you’d have to make sure that an eventual invalid form is handled correctly by get_context_data(). It’s not hard but it certainly isn’t as straightforward as the example above either.

The custom view is the most obvious way of keeping the form instantiation in one place.

Form views

Generic create and update views could look something like this, again reusing the shortcuts mentioned above:

def save_and_redirect_to_object(request, form):
    object = form.save()
    return redirect(object)

def get_form_instance(request, *, model, form_class, instance=None):
    assert model or form_class, "Provide at least one of model and form_class"
    if form_class is None:
        form_class = modelform_factory(model)
    data = (request.POST, request.FILES) if request.method == "POST" else ()
    return form_class(*data)

def object_create(request, *, model=None, form_class=None, form_valid=save_and_redirect_to_object):
    form = get_form_instance(request, model=model, form_class=form_class)
    if form.is_valid():
        return form_valid(request, form)
    return render_detail(request, form.instance, {"form": form}, template_name_suffix="_form")

def object_update(request, *, model, slug, slug_field="slug", form_class=None, form_valid=save_and_redirect_to_object):
    object = get_object_or_404(model, **{slug_field: slug})
    form = get_form_instance(request, model=object.__class__, form_class=form_class, instance=object)
    if form.is_valid():
        return form_valid(request, form)
    return render_detail(request, form.instance, {"form": form}, template_name_suffix="_form")

You want to redirect to a different URL and maybe emit a success message? Easy:

def article_form_valid(request, form):
    form.save()
    messages.success(request, _("Successfully updated the article."))
    return redirect("articles:list")

urlpatterns = [
    ...
    path(
        "<slug:slug>/update/",
        object_update,
        {"model": Article, "form_valid": article_form_valid},
        name=...
    ),
    ...
]

Yes, these generic views wouldn’t allow overriding the case when a form was invalid. But, I’d assume that displaying the form with error messages is the right thing to do in 90% of the cases. And if not, write your own specific or generic view? After all, with the mentioned tools it won’t take up more than a few lines of straightforward code. (If the code was tricky it would be different. But views shouldn’t be tricky.)

Adding more form_valid handlers should be mostly painless. A few examples inspired by Django’s generic editing documentation:

def save_and_redirect_to(url):
    def fn(request, form):
        form.save()
        return redirect(url)
    return fn

def send_mail(request, form):
    form.send_email()
    return HttpResponseRedirect("/thanks/")

def set_author_and_save(request, form):
    form.instance.created_by = request.user
    object = form.save()
    return redirect(object)

You could also couple the form a bit to the request and do something like:

def process_form(request, form):
    return form.process(request)

Sure, forms probably shouldn’t know much about requests. But then, Django is a framework for perfectionists with deadlines and sometimes practicality beats purity.

Date-based generic views

I think I would want to offer a few analyzers which allow easily returning a data structure suitable for rendering links for yearly, monthly, weekly or even daily (who writes that much?) archives. The .dates() queryset method method should be a big help there.

The archive views themselves are straightforward adaptations of the object_list view above.

It may feel like leaving out the actually hard part but I’d have to be convinced that this is actually a hard problem and not just a problem of making basically arbitrary choices which people then adapt to and then think that this is the way things should be since it’s the way things are.

Wrapping up

Some points this post could have made or tried to make are made much better by Luke Plant in the guide Django Views - The Right Way. I don’t generally think that class-based views never make sense. I also don’t think that people shouldn’t use the available tools. I just think that I, myself, don’t want to use them, and I also think that I’m still happier with lambda request: HttpResponseRedirect(...) than with generic.RedirectView.as_view(url=...). The point isn’t to compare the character count. The point is: Does the RedirectView cause a permanent or a temporary redirect? I had to look it up for a long time, and then it changed. The former is completely obvious.

Closing words

I know that people have strong opinions. I’m not interested in all of them. I’m mostly interested in design critiques and arguments regarding the beginner to intermediate skills argument. It’s fine if CBVs work fine for you, and there’s no need to feel challenged by this post.

Thanks for reading!

Also called the WET rule (Write Everything Twice). (Not coined by me.) ↩

How ruff changed my Python programming habits

2023-07-26T12:00:00Z

How ruff changed my Python programming habits

ruff isn’t just a faster replacement for flake8, isort and friends.

With other Python-based formatters and linters there’s always a trade off between development speed (waiting on git commit is very boring) and strictness.

ruff is so fast that enabling additional rules is practically free in terms of speed; the only question is if those rules lead to better, or maybe just to more correct and consistent code.

I have long been using pre-commit, and even longer flake8, black, isort. I have written a piece about flake8 and the value of standards almost 9 years ago and have continued moving in the mentioned direction ever since.

These days I have enabled a wide variety of rules. I’m not sold on all of them (looking at you, pylint) and I’m definitely not of the opinion that rules which I’m not using currently are worthless. I didn’t even know most of these rules before starting to use ruff, and ruff making them easy and painless to use (without a measureable performance penalty) has certainly lead to me annoying my coworkers with a growing set of enabled rules.

Rules

The current ruleset and some justifications for it follows.

pyflakes, pycodestyle

No justification necessary, really.

mccabe

I like the cyclomatic complexity checker, but I have relaxed it a bit. I find it very useful to avoid complex code, but some code is totally straightforward (e.g. building a queryset from a wide variety of query parameters) but still has many if statements. I’d rather allow more complexity instead of sprinkling the code with # noqa statements.

isort

Sorted imports are great.

pep8-naming

Mostly great except when it flags Django’s migration files. The filenames always start with numbers and that’s obviously not a valid Python module name, but it’s not supposed to be.

pyupgrade

pyupgrade is totally awesome.

flake-2020

Avoiding non future-proof uses of sys.version and sys.version_info is a good idea, no questions about that.

flake8-boolean-trap

Sometimes annoying, mostly useful. I don’t like that the plugin flags e.g. with_tree_fields(True) or with_tree_fields(False) because I don’t think this could be possibly misread. But, apart from these edge cases it really is a good idea, especially since keyword-only arguments exist and those aren’t flagged by this rule.

flake8-bugbear

Mostly useful. I have disabled the zip() without strict= warning.

flake8-comprehensions

Checks for unnecessary conversions between generators and lists, sets, tuples or dicts.

flake8-django

I actually like consistency. I also like flagging fields = "__all__", but this check shouldn’t trigger in admin ModelForm classes, really. I probably have to add another entry to [tool.ruff.per-file-ignores] for this.

flake8-pie

Quite a random assortment of rules. I like the no-unnecessary-pass and no-pointless-statements rules, among others.

flake8-simplify

Nice simplifications. I’m not sure if ternary opeartors are always a plus, especially since they hide the branching from coverage.

flake8-gettext

Enormously useful and important. I don’t know how many times I have encountered broken code like gettext("Hello {name}".format(name=name)) instead of gettext("Hello {name}").format(name=name).

pygrep-hooks

Avoids eval(). Avoids blanket # noqa rules (always be specific!)

pylint

I have been using all pylint rules for some time. The pylint refactoring rules (PLR) did prove to be very annoying so I have reverted to only enabling errors and warnings.

The two main offenders were PLR0913 (too many arguments) and PLR2004 (magic value comparison). The former would be fine if it would count keyword-only arguments differently; it’s certainly a good idea to avoid too many positional parameters, I don’t think keyword-only parameters are that bad. The latter is bad because often the magic value is really obvious. If you’re writing code for the web you shouldn’t have to use constants for the 200 or 404 status codes; one can assume that they are well known.

flake8-builtins: Too many boring warnings. I didn’t even want to know that copyright is a Python builtin.
flake8-logging-format: Not generally helpful. Avoiding different exception strings so that e.g. Sentry can group exceptions more easily is a good idea, but the rule generated so many false positives as to be not useful anymore.

Final words (for now)

I really hope that Black is integrated into ruff one day.

Also, I hope that ESLint and prettier will be replaced by a faster tool. I have my eyes on a few alternatives, but they are not there yet for my use cases.

Matthias Kestenholz: Posts about Python

Composition over inheritance: The case for function-based views

Composition over inheritance: The case for function-based views

The early days

The introduction of class-based views

Generic views could be simple

ListView and DetailView

Detail view with additional behavior

Form views

Date-based generic views

Wrapping up

Closing words

How ruff changed my Python programming habits

How ruff changed my Python programming habits

Rules

pyflakes, pycodestyle

mccabe

isort

pep8-naming

pyupgrade

flake-2020

flake8-boolean-trap

flake8-bugbear

flake8-comprehensions

flake8-django

flake8-pie

flake8-simplify

flake8-gettext

pygrep-hooks

pylint

RUF100

Line length

Rules I don’t like

Final words (for now)