avatarMike Huls

Summary

The provided content discusses the use of __slots__ in Python classes to improve performance and memory usage, while also touching on the trade-offs in flexibility and safety.

Abstract

The article "Should You Use Slots? How Slots Affect Your Class, and When and How to Use Them" explains that using __slots__ can significantly enhance the performance of Python classes by up to 20%, as well as reduce memory usage by approximately 60%. Slots work by replacing the dynamic dictionary used by default in Python classes with a fixed-size array, which allows for faster attribute access and lower memory overhead. However, this optimization comes at the cost of flexibility, as it prevents the dynamic addition of attributes to instances of the class. The author, Mike Huls, advocates for the use of slots in scenarios where many instances of a class are created, or when memory is a constraint, and emphasizes the safety benefits of avoiding typos in attribute names. The article also provides practical examples of how to implement slots and mentions the new feature in Python 3.10 that allows the use of slots in dataclasses.

Opinions

  • The author, Mike Huls, believes that the benefits of using slots, such as increased speed and reduced memory usage, outweigh the drawbacks in many cases.
  • Huls suggests that the reduced flexibility of slotted classes can be beneficial for code safety, as it prevents the creation of unintended attributes due to typos.
  • The article implies that slots are particularly useful in applications like video games or physics simulations, where numerous instances with a fixed set of attributes are tracked over time.
  • The author acknowledges that some Python packages may expect non-slotted classes and that developers should be aware of this potential compatibility issue.
  • Huls personally prefers using slots wherever possible, as he rarely needs to create attributes dynamically and appreciates the static nature of slotted classes.
  • The author encourages readers to experiment with slots and provides links to additional resources and articles for further reading and learning.

Should You Use Slots? How Slots Affect Your Class, and When and How to Use Them

One line of code for a 20% performance increase?

(image by Sébastien Goldberg on Unsplash)

Slots are a mechanism that allow you to declare class attributes and restrict the creation of other attributes. You establish which attributes your class has, preventing developers from adding new attributes dynamically. This generally leads to a 20% speed increase.

Slots are especially beneficial in programs where you have a large number of class instances with a known set of attributes. Think of a video games or physics simulations; in these situations you track a large number of entities over time.

You can add slots to your class adding a single line of code but is this always a good idea? In this article we’ll look a why and how using slots make your classes that much faster and when to use them. The overall goal is to better understand how Python’s class internals work. Let’s code!

Slots make Python classes faster

You can improve a class’ memory usage and performance by making it use slots. A class with slots takes up less memory and executes faster.

How to make my class use slots?

Telling Python to make a class use slots is very simple. You just add a special attribute called __slots__ that specifies the names of all other attributes:

class Person:
  first_name:str
  last_name:str
  age:int
  
  __slots__ = ['first_name', 'last_name', 'age']    # <-- this adds slots

  def __init__(self, first_name:str, last_name:str, age:int):
    self.first_name = first_name
    self.last_name = last_name
    self.age = age

In the class above we see that Person has three attributes: first_name, last_name, and age. We can tell Python that we want the Person class to use slots by adding the __slots__ attribute. This attribute has to specify the names of all other attributes.

How much faster are slotted classes?

The Person class we’ve used above is almost 60% smaller using slots (488 bytes to 206 bytes).

With regards to speed, I’ve benchmarked instantiation, accessing and assignment. I’ve found speed increases up to 20%! You have to take these results with a grain of salt; though these percentages seem pretty impressive these 20% represent only 0.44 seconds for instantiation the class 10 million times. This comes down to a negligible 44 nanoseconds per instance (roughly 30.3 million times smaller than a second).

See code for benchmarking memory and speed;

Why are slotted classes smaller and faster?

This has to do with Python classes’ dynamic dictionary. This dictionary lets you assign attributes to Python classes:

class Person:
  pass

mike = Person()

mike.age = 33  # <-- create a new attribute

In the example above we define a class without any attributes, create an instance of that and then dynamically create the age attribute and assign it a value.

Under the hood Python stores all attribute information in a dictionary. This dictionary is available by calling the __dict__ magic method on the class:

# 1. Define class
class Person:
  name:str
  
  def __init__(self, name:str):
      self.name = name

# 2. Create instance
mike = Person(name='mike')
# 3. Create a new variable
mike.age = 33
# 4. Create new attribute throught the __dict__
mike.__dict__['website'] = 'mikehuls.com'
# 5. Print out the dynamic dictionary
print(mike.__dict__)  
# -> {'name': 'mike', 'age': 33, 'website': 'mikehuls.com'}

The dynamic dict makes Python classes pretty flexible but it has a downside: using the attributes makes Python search in this dict, which is relatively slow.

How do slots affect the dynamic dict?

When you tell Python to slot your class, the dynamic dict is not created. Instead Python creates a fixed-size array that contains the references to your variables. This is why you have to pass the names of your attributes to the __slots__ attribute.

Not only is accessing this array much faster, it also takes up less memory space. A smaller memory footprint also has beneficial effects on memory allocation and garbage collection.

What are the side effects of slots?

Slots change your class; it becomes a bit more inflexible since your class become a bit more static. This means that you cannot add attributes at runtime; you have to specify your attributes beforehand:

# 1. Define class
class Person:
  name:str
 
  def __init__(self, name:str):
      self.name = name

# 2. Create instance
mike = Person(name='mike')

# 3. Add a new attribute?
mike.website = 'mikehuls.com'     # this will not work!
# ERROR: AttributeError: 'Person' object has no attribute 'website'

# 4. Print out dynamic dict
print(mike.__dict__)              # this will not work
# ERROR: AttributeError: 'Person' object has no attribute '__dict__'

There is a (albeit bit messy) way around this: by adding the value "__dict__" to your __slots__ array:

# 1. Define class
class Person:
  name: str

  __slots__ = ["name", "__dict__"] # <- We've added __dict__

  def __init__(self, name: str):
    self.name = name

# 2. Create instance
mike = Person(name='mike')

# 3. Add a new attribute
mike.website = 'mikehuls.com'     # no error this time!

The last thing to keep an eye on is the fact that some packages may expect “normal” Python classes in stead of slotted ones.

Does this also work with dataclasses?

Yes! Starting from Python 3.10 you can also add slot dataclasses. It’s even easier with dataclasses, just add a single argument to the @dataclass decorator. Just define your dataclass like below:

@dataclasses.dataclass(slots=True)
class Person:
    name: str

What are the advantages of using slots?

Obviously the speed and memory efficiency but maybe also safety: if I want to overwrite the age attribute on my class but make a typo and type mike.aage = 34 then unslotted classes will just create a new attribute, keeping the age attribute unchanged. When you use slots Python will throw an error because it doesn’t know an aage attribute on that class.

When to use slots?

SPEED: Although slots speed up you class percentage-wise, the absolute time increase is pretty negligible per operation. Therefore slots become more attractive to use if you have to create a lot of instances, or have to overwrite or access attributes many, many times.

MEMORY: If you’re low on memory and can save every byte it may be beneficial to use slots since they cut the amount of memory used significantly. Our simple class was reduces by 60%.

SAFETY: Slots prevent you from using wrong attributes and creating new attributes dynamically. Slotted classes throw an error if you try to modify an unknown attribute.

Conclusion

As we’ve seen in this article slots affect your classes in three ways:

  • SIZE: slots eliminate the need for Python to create the dynamic dictionary but instead relies on a smaller, fixed-size array, which indirectly speeds up your app by decreasing demand on garbage collection for example.
  • SPEED: slots allow for accessing the memory directly, bypassing the need to search the dictionary, which is much slower. Speed improvements are pretty marginal in an absolute sense; saving a few nanoseconds.
  • FLEXIBILITY: slots prevent adding attributes at runtime so your classes become a bit less flexible. This can also be a good thing since your code may get messy when you use dynamic attribute creation.

In my opinion, reduced flexibility is a downside I don’t experience often: I never create attributes dynamically and I like that slots keep the attributes static. Therefore I use slots wherever possible. In the worst case a dependency trips up but in that case it’s very easy to remove the slots again.

I hope this article was as clear as I hope it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of programming-related topics like these:

Happy coding!

— Mike

P.S: like what I’m doing? Follow me!

Programming
Coding
Python
Software Development
Data Science
Recommended from ReadMedium