What Are Python Special Methods and Their Purpose?

[fog37]

TL;DR

Understand the difference between special methods and non-special methods

Hello Forum,

In Python, methods are functions which are defined inside a class. Python has many reserved keywords and built-in functions like print(), add(), type(), etc that we commonly use even when we don't create classes, instances, etc. Whenever we use an operator (like + or - ) or a built-in function like print() or int(), a special method is automatically executed behind the scenes.

• In Python, each different type of object (integer, string, function, etc.) has a number of possible special methods that can be called and applied to the object itself based on its type. Special methods are predefined methods (not user-defined methods) can be used inside a user-defined class. I believe that special methods essentially allow us to extend the capabilities of built-in functions/methods so they can be applied to objects that these built-functions/methods are not designed for. Is that correct? For example, dir(5) shows which methods can be used on objects that are integers but not strings. But it may be possible to use one of the listed methods reserved for integers also on strings within a class if we include the special method itself.

• When we create a class and include the method __init__ to initialize the class, the method __init__ automatically runs to create class attributes (the characteristics of any instance of the class) whenever we create an instance from the class. Are all other special methods that we can include in a class definition also executed automatically? On the other hand, non-special methods must be called using the dot notation.
• Other examples of special methods are __main__ , __str__ , __new__ What does the special method __main__ exactly do?
• What about the special method __str__ , which is a method used to convert objects to strings in the context of a class? I think __str__ provides a string representation of the objects generated from the class...Does it automatically get executed whenever we create a class instance generating the name of the object on the screen?

Thanks!

 

[wle]

Special methods are predefined methods (not user-defined methods) can be used inside a user-defined class.

Most "special methods" are not predefined*. You define them yourself just like any other method. The only thing "special" about them is their names and that there are conventions in Python about when methods with those names will be called on your behalf if you defined them. For example, when you create a new class instance the Python interpreter will call the __init__() method you defined for it, if you defined one. If you evaluate x + y and x is not one of the built-in numeric types then Python will evaluate x.__add__(y) using its __add__() method, if it has one.

*More precisely: Python has a class hierarchy in which all classes are direct or indirect subclasses of (and thus directly or indirectly inherit attributes from) a special class called object. object defines default implementations of a few of the special methods (__init__() is one of them), which you can redefine in your own classes if you want, but doesn't give default definitions of all of them.

Are all other special methods that we can include in a class definition also executed automatically?

They are executed automatically in the sense that there are conventions defined in Python about when they will be called. With __init__() this is just after the object is created. The calling of other special methods is triggered in different situations, sometimes when you use special syntax (e.g., obj1 + obj2 calls obj1.__add__(obj2)) and sometimes in standard functions (e.g., str(obj) calls obj.__str__()). These let you effectively extend the capabilities of Python's syntax and some of its standard library.

Other examples of special methods are __main__ , __str__ , __new__ What does the special method __main__ exactly do?

__main__ is not the name of a special method (not everything with a name that starts and ends with underscores is). It is the name of the top-level module your code starts running in when you run a Python script or start an interactive session: https://docs.python.org/3/library/__main__.html.

Python's special method names are documented here: https://docs.python.org/3/reference/datamodel.html#specialnames.

What about the special method __str__ , which is a method used to convert objects to strings in the context of a class? I think __str__ provides a string representation of the objects generated from the class...Does it automatically get executed whenever we create a class instance generating the name of the object on the screen?

__str__() is one of two special method names that let's you define standard or default string representations for objects. The other one is __repr__(). They are called when you call the built-in str() and repr() functions respectively and in other situations where you implicitly ask to generate a string representation for an object, including the print() function and the string format() method. __repr__() is also used by the Python REPL (read-eval-print loop) to generate the output you see when you evaluate code in an interactive Python shell.

The difference between the two methods is that __str__() is intended to return an "aesthetic" (nice for humans to read) representation of an object while __repr__() is meant to return a "source code" representation of an object. In other words, if x is an object then ideally you should define __repr__() in such a way that you can copy-and-paste the output of print(x.__repr__()) into the interactive Python shell (or run eval(x.__repr__()) and it will make an object that is a copy of x.

Here's some examples that illustrate this with Python strings and Fraction objects:

>>> 'hello'
'hello'             # <-- This output is generated by calling
                    #     'hello'.__repr__() and printing the result.
>>> 'hello'.__str__()
'hello'
>>> 'hello'.__repr__()
"'hello'"           # <-- Notice the extra quotes.
>>> print('hello')
hello               # <-- print() calls the object's __str__() method and
                    #     prints the contents of the string.
>>> print('hello'.__repr__())
'hello'             # <-- We could copy & paste this output into Python code
                    #     to create a string.
>>> from fractions import Fraction
>>> x = Fraction(3, 2)
>>> x
Fraction(3, 2)      # <-- Code we could type to create the same fraction.
>>> x.__repr__()
'Fraction(3, 2)'
>>> x.__str__()
'3/2'               # <-- More human-friendly representation of a fraction.
>>> print(x)
3/2                 # <-- print() prints the __str__() representation of an
                    #     object.
>>> print('{!r}'.format(x))
Fraction(3, 2)      # <-- You can use the !r format directive to say you want
                    #     the __repr__() representation of an object.
>>> eval(x.__repr__())
Fraction(3, 2)      # <-- eval(x.__repr__()) ends up creating a new fraction
                    #     object representing the same fraction as x.
>>> eval(x.__str__())
1.5                 # <-- This time we get a floating-point number (the
                    #     result of evaluating 3/2 in Python) instead of a
                    #     fraction object.

 

[fog37]

Most "special methods" are not predefined*. You define them yourself just like any other method. The only thing "special" about them is their names and that there are conventions in Python about when methods with those names will be called on your behalf if you defined them. For example, when you create a new class instance the Python interpreter will call the __init__() method you defined for it, if you defined one. If you evaluate x + y and x is not one of the built-in numeric types then Python will evaluate x.__add__(y) using its __add__() method, if it has one.

Thank you wle! Quick question: not everything that has double underscore is special method. There are objects like variables (ex: __docstr__) or modules that have the double underscore...So, in general, what does the dunder (before and after the name) indicate for a method, variable or module?

There are even objects with single underscores in Python (name mangling).

 

[PeterDonis]

@fog37 please do not quote someone's entire post in your response. Only quote the part you are responding to.

 

[wle]

So, in general, what does the dunder (before and after the name) indicate for a method, variable or module?

By itself, nothing, as far as I know. The underscore is just a character that you can use or not use in variable or method names just like many other characters.

This is what PEP 8 -- Style Guide for Python Code says about method names:

__double_leading_and_trailing_underscore__: "magic" objects or attributes that live in user-controlled namespaces. E.g. __init__, __import__ or __file__. Never invent such names; only use them as documented.

Probably the aim was just to use unusual names that you would be unlikely to use yourself to avoid name collisions as well as make it clear when a method is special. So if you read a method name __doit__() in some Python code then this is a hint that there is something special about this method and you may want to consult the documentation to find out what. On the other hand, if you call a method doit() in your own code then you can be sure that 1) you're not accidentally using the name of a special method and you don't need to open the Python documentation to check this, since you know they all start and end with double underscores, and 2) it also won't become a special method name in a future version of Python.