A Comprehensive Guide into Python Descriptors

This article will delve into an advanced feature of Python known as descriptors. Although descriptors can be somewhat complex, understanding them is made easier by taking a step-by-step approach. That’s why we will begin with some simple code examples and then gradually refactor them, adding layers of understanding as we progress.

Starting with a Problem

Imagine that we need to create a Python class representing an item in a store. This item would have a few attributes: its name, inventory, and price. Additionally, we want to implement some validation rules for these attributes:

• The price and inventory must never be less than 0.
• The name must be at least 3 characters long.

In the sections that follow, we will explore how Python descriptors can help us enforce these rules elegantly and efficiently.

The Ugly Method

Let’s dive into our very first iteration of solving the problem. I must warn you — this solution is going to be ugly! But it will serve as a foundation for understanding what Python descriptors can offer.

class Product:
    def __init__(self, name, price, weight):
        self._name = None  #1
        self._price = None
        self._weight = None
        self.set_name(name)  #2
        self.set_price(price)
        self.set_weight(weight)

    def set_name(self, name):  #3
        if len(name) < 3:
            raise ValueError("Value must be at least 3 characters")
        self._name = name
    
    def get_name(self):  #4
        return self._name

    def set_price(self, price):
        if price < 0:
            raise ValueError("Value must be positive")
        self._price = price

    def get_price(self):
        return self._price

    def set_weight(self, weight):
        if weight < 0:
            raise ValueError("Value must be positive")
        self._weight = weight

    def get_weight(self):
        return self._weight

>>> product = Product("Milk", 2, 3)
>>> product.get_name()
Milk
>>> product.set_price(1.5)
>>> product.get_price()
1.5
>>> product.set_price(-2.5)
Traceback (most recent call last):
ValueError: Value must be positive

1. We initially set self._name , self._price and self._weight to None .
2. We then call the setter methods to assign values to the respective attributes, using validation to ensure that the inputs meet our requirements.
3. Each attribute has its own dedicated setter method to handle validation.
4. Each attribute also has a corresponding getter method to retrieve its value.

Although this class provides dedicated methods for setting and getting values and uses the underscore prefix to suggest that these attributes are private, it’s far from an elegant solution. Let’s see why:

Issues with This Code

• It’s Not DRY (Don’t Repeat Yourself): There’s repetition in validation code, making it more cumbersome to maintain.
• Bulky Code: For such simple validation, we’ve ended up with a lot of methods, making the code harder to read and understand.
• Lack of Intuition: Unless you delve into the code or read extensive documentation, it’s not obvious that you need to use dedicated methods for setting and getting attributes.
• Unnatural Usage: If you wanted to update the price, for example, you might expect to simply set product.price = 0.5. However, with this design, you must instead call product.set_price(0.5).

Refactoring using the Property Class

In our previous example, we manually created setters and getters by defining individual methods. Now, let’s see how we can improve this by using Python’s property class, which is indeed a class, not a function!

Introducing the Factory Class

We discussed earlier how unintuitive our solution was, with dedicated get_something and set_something methods for each attribute. The property class can help us make these methods more user-friendly. Let's see an example:

class JustANumber:
    def __init__(self, num):
        self._num = num

    def get_num(self):
        return self._num

    def set_num(self, new_num):
        self._num = new_num

    num = property(get_num, set_num)

In this code, we’ve defined a property instance that accepts getter and setter functions. When you access or update num, it will actually execute get_num and set_num, respectively.

The “Old” Way

>>> inst = JustANumber(42)
>>> instance.get_num()
42
>>> instance.set_num(45)
>>> instance.get_num()
45
>>> instance.set_num(instance.get_num() + 2)
>>> instance.get_num()
47

The Proper(ty) Way

>>> instance = JustANumber(42)
>>> instance.num
42
>>> instance.num = 45
>>> instance.num
45
>>> instance.num += 2
>>> instance.num
47

This approach is already more intuitive. But we can simplify it further:

class JustANumber:
    def __init__(self, num):
        self._num = num

    @property
    def num(self):
        return self._num

    @num.setter
    def num(self, new_num):
        self._num = new_num

The property class can be used as a decorator. Byusing the pattern above, we’ve made our code even more concise.

Refactoring Our Solution

Now that we understand how to use the property class, let's refactor our Product class.

def positive_num_validation(num):  #1
    if num < 0:
        raise ValueError("Value must be positive")

def min_char_len(s, length):
    if len(s) < length:
        raise ValueError(f"Value must be at least {length} characters")


class Product:
    def __init__(self, name, price, weight):
        min_char_len(name, 3)  #2
        self._name = name
        positive_num_validation(price)
        self._price = price
        positive_num_validation(weight)
        self._weight = weight

    @property  #3
    def name(self):
        return self._name

    @name.setter  #4
    def name(self, name):
        min_char_len(name, 3)
        self._name = name

    @property
    def price(self):
        return self._price

    @price.setter
    def price(self, price):
        positive_num_validation(price)
        self._price = price

    @property
    def weight(self):
        return self._weight

    @weight.setter
    def weight(self, weight):
        positive_num_validation(weight)
        self._weight = weight

1. We now have a helper function that raises an error if the value provided is less than 0.
2. The validations are run when the class is initialised.
3. The name method is decorated with property . This registers this method as the getter.
4. Registering the setter, this method is therefore called whenever the attribute is updated.

Issues with This Code

Though this code is an improvement, there are still issues that can be considered “code smells”:

• Validation must be executed manually in both the initialiser and the setter methods.
• The getter methods all do the same thing — return different attributes — breaking the DRY principle.
• The setter methods all do the same thing — execute validation and update a value — again, not adhering to DRY.

In the next section, we will explore how Python descriptors can further refine our code and address these concerns.

Refactoring Using Python Descriptors

How Python Descriptors Work

Python descriptors allow us to manage the access to an object’s attributes in a flexible way. They work by defining methods like __get__ and __set__ in a class, which is then used to manage a specific attribute in another class. Let's take a look at a simple example (taken from the Python docs):

class Ten:
    def __get__(self, instance, owner):
        return 10


class A:
    ten = Ten()

>>> inst = A()
>>> inst.ten
10

In this example, the __get__ method of Ten is executed whenever we access a.ten. The __get__ method has two parameters, instance and owner, which respectively refer to the instance (inst) and class (A) that use the descriptor.

Now, let’s add the ability to set values:

class FunkyNumber:
    def __get__(self, instance, owner):
        return instance._number * 100  #1

    def __set__(self, instance, value):
        instance._number = value - 1  #2


class A:
    number = FunkyNumber()

    def __init__(self, num):
        self.number = num

>>> a = A(5)
>>> a.number
400
>>> a._number
4
>>> a.number = 10
>>> a.number
900

In this case, when we set a.number, the __set__ method in FunkyNumber is invoked, setting a._number to value - 1. Then, when we get a.number, a._number is multiplied by 100.

Creating a PositiveNum Descriptor

Now, let’s create a descriptor that ensures the value set is positive:

class PositiveNum:
    def __get__(self, instance, owner):
        return instance._value

    def __set__(self, instance, value):
        if value < 0:
            raise ValueError("Value must be positive")
        instance._value = value

This descriptor can be used in the Item class:

class Item:
    price = PositiveNum()

    def __init__(self, price):
        self.price = price

Let’s see this in action:

>>> item = Item(5)
>>> item.price
5
>>> item.price = 10
>>> item.price
10
>>> item.price = -1
Traceback (most recent call last):
ValueError: Value must be positive
>>> item = Item(-5)
Traceback (most recent call last):
ValueError: Value must be positive

Our validation works! We no longer need to explicitly call the validation in both the initialiser and the setter method.

Adding Flexibility to a Python Descriptor Using __set_name__

In our implementation of PositiveNum , when we set the value, we update _value in the instance itself. But what happens if our class using the descriptor now becomes:

class Item:
    price = PositiveNum()
    weight = PositiveNum()

    def __init__(self, price, weight):
        self.price = price
        self.weight = weight

Now we have a problem. As both price and weight reference _value , we cannot reuse the descriptor. This isn’t ideal. Let’s explore how we can make our descriptor more flexible:

class PositiveNum:
    def __set_name__(self, owner, name):  #1
        self.storage_name = name

    def __set__(self, instance, value):
        if value < 0:
            raise ValueError("Value must be positive")
        instance.__dict__[self.storage_name] = value  #2

1. The __set_name__ method is called when the descriptor is initialised. When this is done, the name argument is populated with the name of the variable holding a reference to the descriptor instance. So, where we have price = PositiveNum() , the name will be "price" .
2. Instead of updating the instance._value = ... , we now update the __dict__ object directly in the instance. For those who are a little confused about this, every class implements an __dict__ object which stores all class attributes. You may be wondering why not use setattr ? The problem with using setattr(instance, self.storage_name) here is that it would recursively call the same __set__ method over and over again. And so, we need to update the __dict__ object directly.

We now have a more flexible descriptor that we can reuse.

Let’s now refactor our solution.

Refactoring Using Python Descriptors

class PositiveNum:
    def __set_name__(self, owner, name):
        self.storage_name = name

    def __set__(self, instance, value):
        if value < 0:
            raise ValueError("Value must be positive")
        instance.__dict__[self.storage_name] = value


class MinLenChar:
    def __init__(self, length):
        self._length = length

    def __set_name__(self, owner, name):
        self.storage_name = name

    def __set__(self, instance, value):
        if len(value) < self._length:
            raise ValueError(
                f"Value must be at least {self._length} characters"
            )
        instance.__dict__[self.storage_name] = value


class Product:
    name = MinLenChar(3)
    price = PositiveNum()
    weight = PositiveNum()


    def __init__(self, name, price, weight):
        self.name = name
        self.price = price
        self.weight = weight

>>> product = Product("Milk", 2, 43)
>>> product.name
Milk
>>> product.price
2
>>> product.price = 5
>>> product.price
5
>>> product.price = -5
Traceback (most recent call last):
ValueError: Value must be positive

We’ve now refactored our code to use Python descriptors. The Product class is far cleaner and easier to read. In addition, we also have a couple of reusable classes that we can use elsewhere.

Python descriptors are a powerful tool that can be used to make your code more maintainable and readable. It is in fact a key building block used by the Django web framework in their models.

You may feel at this point that this code can still be refactored further. As this post is already quite lengthy, we won’t be exploring further refactoring in depth. Below is however my final version of this code, I would love to see if you have any other interesting ways to refactor the code!

from abc import ABC, abstractmethod
from dataclasses import dataclass


class ValidationBase(ABC):
    @abstractmethod
    def validate(self, value):
        pass


class ValidationDescriptor(ValidationBase):
    def __set_name__(self, owner, name):
        self.storage_name = name

    def __set__(self, instance, value):
        self.validate(value)
        instance.__dict__[self.storage_name] = value


class PositiveNum(ValidationDescriptor):
    def validate(self, value):
        if value < 0:
            raise ValueError("Value must be positive")


class MinLenChar(ValidationDescriptor):
    def __init__(self, length):
        self._length = length

    def validate(self, value):
        if len(value) < self._length:
            raise ValueError(
                f"Value must be at least {self._length} characters"
            )


class Product:
    name = MinLenChar(3)
    price = PositiveNum()
    weight = PositiveNum()

    def __init__(self, name, price, weight):
        self.name = name
        self.price = price
        self.weight = weight

Conclusion

The Python descriptors mechanism provides a way to enforce attribute behaviour, encapsulation, and more. This powerful feature can be leveraged to make your code more readable, maintainable, and reusable, as demonstrated in the examples.

The refactored code even uses abstract classes to generalise validation logic, which can easily be extended for various other validation requirements.

In frameworks like Django, descriptors are indeed a key building block used in models, showing how important and useful this feature can be in real-world applications.

Feel free to explore this concept further and come up with your unique ways of utilising Python descriptors. The possibilities are vast, and the power of abstraction they offer can lead to even more elegant and efficient solutions in your programming tasks.

In Plain English

Thank you for being a part of our community! Before you go:

• Be sure to clap and follow the writer! 👏
• You can find even more content at PlainEnglish.io 🚀
• Sign up for our free weekly newsletter. 🗞️
• Follow us on Twitter, LinkedIn, YouTube, and Discord.