Some pylint tips

I’ve been using pylint for quite some time now, so today I’d like to share a few tips with you.

Update: Itamar Turner-Trauring also wrote a nice article about pylint, you should read it too: Why Pylint is both useful and unusable, and how you can actually use it

What is pylint? #

It’s a static analyzer for Python code. “Static” means that it won’t execute your code, it will just parse it to find mistakes or things that do not respect a given coding style.

Pylint is capable of emitting very interesting warnings.

Here are some examples:

def foo(bar, baz):
    pass

foo(42)

class MyThread(threading.Thread):
    def __init__(self, name):
        self.name = name

    def run(self):
        ...


my_thread = MyThread()
my_thread.start()

This is nice because it saves you a fatal assertion at runtime:

AssertionError: Thread.__init__() not called

Fixing pylint output #

By default, pylint is very verbose:

$ pylint my_module.py
No config file found, using default configuration
************* Module my_module
C:  1, 0: Missing module docstring (missing-docstring)
...

Report
======
8 statements analysed.

Statistics by type
------------------

+---------+-------+-----------+-----------+------------+---------+
|type     |number |old number |difference |%documented |%badname |
+=========+=======+===========+===========+============+=========+
|module   |1      |1          |=          |0.00        |100.00   |
+---------+-------+-----------+-----------+------------+---------+
...
+---------+-------+-----------+-----------+------------+---------+


Raw metrics
-----------

...

Messages by category
--------------------

+-----------+-------+---------+-----------+
|type       |number |previous |difference |
+===========+=======+=========+===========+
|convention |3      |3        |=          |

...

+-----------+-------+---------+-----------+
|error      |1      |1        |=          |
+-----------+-------+---------+-----------+


Global evaluation
-----------------
Your code has been rated at -3.75/10 (previous run: 1.43/10, -5.18)

Let’s fix that!

The goal is to have no output at all when everything is fine, only have the errors if something is wrong, and make sure output of pylint can then be used by an other program if required. ¹

Get rid of “No config file found, using default configuration” #

Just go to the root of your project and run:

$ pylint --generate-rcfile > pylintrc

More readable output #

Edit the pylintrc file to have:

[REPORTS]

output-format=parseable

That way you’ll get a more standard output, with the file name, a colon, the line number and the error message:

my_module.py:11: [C0103(invalid-name), ] Invalid constant name "my_thread"

This is useful if you want to run pylint from your editor and quickly jump to the lines that contains errors.

Get rid of the useless stuff #

You don’t really care about all the stats, so let’s just disable everything:

[REPORTS]

reports = no

There! Now we only get only the warning or error messages, except there’s a big line to separate the modules:

************* Module my_module
my_module.py:4: [W0231(super-init-not-called), MyThread.__init__] __init__ method from base
                class 'Thread' is not called
************* Module other_stuff
other_stuff.py:5: [W0311(bad-indentation), ] Bad indentation. Found 3 spaces, expected 4

Correcting false positives #

Sometimes pylint thinks there’s a problem in your code even though it’s perfectly fine.

Here’s an example using the excellent path.py library:

my_path = path.Path(".").abspath()
my_path.joinpath("foo")

If you take a look, it seems that pylint gets confused by upstream code:

class multimethod(object):
    """
    Acts like a classmethod when invoked from the class and like an
    instancemethod when invoked from the instance.
    """
    ...


class Path():
    ...

    @multimethod
    def joinpath(cls, first, *others):
        if not isinstance(first, cls):
            first = cls(first)
        return first._next_class(first.module.join(first, *others))

It’s some dark magic (yeah Python) to make sure you can use both:

path_1 = my_path.joinpath("bar")
path_2 = path.Path.joinpath(my_path, "bar")

and pylint only “gets” the second usage…

Here the solution is to use a “pragma” to tell pylint that the code is fine

my_path = path.Path(".").abspath()
# pylint: disable=no-value-for-parameter
a_path = my_path.joinpath("foo")

But if you do that, you’ll get:

The solution is to disable warnings about disabled warnings (so meta):

[MESSAGES CONTROL]

disable=reduce-builtin,dict-iter-method,reload-builtin, ... ,locally-disabled

Freezing pylint version #

If you’re like me, you probably have a dev-requirements.txt containing a line about pylint in order to use in a virtualenv.

It’s also possible you’re using buildout.

But anyway, I highly recommend you have a separate installation of pylint just for your project, isolated from the rest of your system.

The fact is that pylint depends on astroid, and both projects are constantly evolving.

So if you’re not careful, you may end up upgrading astroid or pylint and suddenly some false positives will get fixed, and some other will appear.

So to make sure this does not happen, always freeze pylint and astroid version numbers, like so:

pylint==1.5.5
astroid==1.4.7

(you can use pip freeze to see the version of all the packages in your virtualenv)

Running pylint #

To run pylint, you have to give it a list of packages or modules do check on the command line.

For instance, let’s assume your sources look like this:

src
  pylintrc
  bar.py
  spam
     __init__.py
     eggs.py

Then you have to call pylint like this:

$ cd src
$ pylint bar.py spam

You may try to run pylint . or just pylint but it won’t work :/

So this means that anytime you add a package or a new module, you have to change the way you call pylint.

This is rather annoying, that’s why I suggest you use invoke.

Write a tasks.py file looking like:

import path
import invoke

def get_pylint_args():
    top_path = path.Path(".")
    top_dirs = top_path.dirs()
    for top_dir in top_dirs:
        if top_dir.joinpath("__init__.py").exists():
            yield top_dir
    yield from (x for x in top_path.files("*.py"))


@invoke.task
def pylint():
    invoke.run("pylint " + " ".join(get_pylint_args()), echo=True)

And then you just have to use:

$ invoke pylint

Speeding up pylint #

pylint also knows how to use multiple jobs so that it runs faster.

Since you are already using tasks.py, you can specify the number of jobs to use in a cross-platform way easily:

import multiprocessing


def get_pylint_args():
    ...

    num_cpus = multiprocessing.cpu_count()
    yield "-j%i" % num_cpus

Shorter version #

You can also not do all of the above and just fire up pylint like so:

$ pylint -E *.py

It will only show errors, (-E is short for --errors-only), and will have a good enough output.

• Pros: you don’t have to write any pylintrc file
• Cons: you may hide serious bugs

That’s all for today, see you next time!