Starting a Python Project

We have all been there: we need to start a new project. It can be an actual project or, at some point in our lives, it could have been a small homework assignment. Now sitting in front of an empty directory, we figure we will not bother with the boilerplate yet.

We hack together a couple of modules, do quick manual testing because, heck, setting up an actual testing infrastructure is boring and takes time. And eventually, we either resolve to spend the time to do it, or for a short-term project, the code remains a hackish mess rotting away which may be fine but a little unsatisfactory.

This is why starting with a nice project template is a good idea.

What do we need?

We have a couple of basic tools to support any project. Roughly, we need:

• A proper code layout, that works well with common Python ecosystem.
• A version control system.
• An automated testing suite.
• Test coverage to assess our tests' quality.
• Standard packaging tools.

And the ability to setup all this quickly and consistently, be it for other contributors or ourselves two years from now.

Lastly, if we are to share and accept contributions, on top of that we need at least:

• Documentation tools to make contributions easier and more in line with our style.
• Continuous integration to check contributions.

That should cover the most common needs.

A simple template

Tools

Unless we have specific, unusual needs, it is best to stick to industry standards, as they are more likely to stand the test of time.

• Git is the obvious choice for new projects outside of established ecosystems. It is ubiquitous, and github has a neat pull-request workflow that works great for projects following an open-source approach (whether they actually are open source or not).
• We will use the pytest framework for testing. Although not part of python as unittest is, it features better support for many third-party libraries and cleaner setup.
• We will use coverage.py to ensure our tests are thorough. Integrates well with pytest, using pytest-cov.
• We will add a tox configuration to make it easy to run tests on several python versions.
• Pylint will ensure we follow PEP 8 style guidelines.
• We will use the sphinx documentation generator that has grown standard in the python community.
• We will use the usual setuptools used in most python projects.

Continuous integration depends a bit more on the environment. If building an open-source project, travis-ci has widespread use. Let's use it.

I will not explain how to use the tools. See the link to the template below.

Layout

I uploaded the template to github. It assumes a project named example. File layout is straightforward:

docs /

The template uses autodoc to extract documentation from docstrings. This directory simply describes the structure of the documentation, importing extracted docstrings in the correct spots.

<project name> /

The namespace of the project. It should have the same name. All the code lives here, and all imports should therefore read like from example.foo import bar. Always use full-path imports.

tests /

Unit tests. Every file in <project name> should have a matching file in tests, prefixed with test_. This makes pytest recognize them automatically. In them, every test case should be a function with a descriptive name starting with test_ as well.

setup.py

Packaging configuration. It defines project dependencies and customizes install. If the project should be runnable, this is where entry points are configured. If you use the template, be sure to update author name and project url there.

Other files should be obvious by just looking at them in the template.

This is it. No excuses for not having a decent setup now. The template is here, setup instructions included. Feel free to copy it, use and reuse it.

More tips and resources

In no particular order:

• The pyscaffold project has a nice generator that creates project templates for specific frameworks.

• If building an asynchronous project using asyncio:

  • add pytest-asyncio and sphinxcontrib-asyncio to requirements-dev.txt.
  • add 'pytest-asyncio' into the tests_require section of setup.py.
  • add 'sphinxcontrib.asyncio' into the extensions section of docs/conf.py.
• If you host your code on github, enabling travis takes less than 5 minutes and it runs your tests at every push automatically.

Happy coding!