Laying the Groundwork

Home » cryptomate series

Initial design is done. It is now time to start laying the basis of the project. At this point, there are a few technical choices we can no longer defer.

Technical decisions

Language choice

As per our architecture constraints, we must choose either C++ or Python, or a mix thereof. I personally tend to favor Python, as a more flexible language, which allows building software much faster than C++, albeit at the cost of performance.

I will stick to this choice here. The initial design will be full python and then, as we identify bottlenecks, we will convert to C++ piece-wise. Especially, as we will have a lot of input/output, python 3.5 asyncio module will be very convenient and allow us to reach an initial working version much faster than C++.

Thus, we will build the initial version using python, with support for version 3.5 and 3.6. And when potential benefit arises, we will use C++14, which has a good balance of modern features and compiler support.

Toolbox

We have a couple of basic tools we will need to support our project. Without ceremony, here are the essential tools we will use:

  • We will use github to host our project. It couples the ubiquitous git versioning system with a neat pull-request workflow that works great for open-source projects.
  • We will use Pylint to identify code smells and style mistakes.
  • We will use the pytest framework for testing.
  • We will use coverage.py to ensure our tests are thorough.
  • We will use travis-ci continuous integration service to ensure our tests are run for every change we make.
  • We will use the sphinx documentation generator.

Those tools come a little from personal preference, and a lot from being industry standards. I will not explain the setup here, but the “starting a python project” post covers the details.

Our newly created repository is on github, and you can check the initial commit. It is now time to look at our directory hierarchy.

Cross-cutting dependencies

Cryptomate has multiple modules with similar requirements, so to keep its parts consistent, we decide to choose some dependencies at the global level.

Event loop

We need a single event loop component. Handling tick data from many markets will put a lot of stress on our I/O loop. Also, as we plan to move modules from Python to C++ as we go, an event loop readily available in both languages would be great.

We shall use libuv, with the uvloop python bindings. It is widely used, and renowned for its performance.

Cross-language integration

In order not to restrict Python-C++ interactions to integration boundaries, we need a standard way to make them talk.

We shall initially use Cython for this purpose. Though not as featureful as Boost.Python or the more recent pybind11, it integrates better within a Python workflow and produces leaner modules. Should we find ourselves moving large parts of the code over to C++, we might reconsider this choice later on.

Mapping the model to files

As per the traditional python model, we will adopt a package-by-module approach. Every module defined in the modules overview shall have its own, possibly nested, directory structure, reflecting its own subdivisions in sub-modules and internal layers.

To enforce isolation and encapsulation:

  • Modules may only access the root of other modules, and cannot reach into the hierarchy of their sub-modules.
  • Modules may only access other modules if allowed to in the architecture. In particular, remember that adapters depend on ports and not the other way around.

One consequence of this approach is that layer boundaries end up within each module. For instance, the real-time market module will contain all objects related to fetching market data in real-time:

  • The description of the port-adapter relationship.
  • The various adapters.
  • The registry that makes it possible to find and instantiate them.
  • The management of their life-cycle.
  • The interfaces that abstract out and expose the functionality.

Another way to view this structure is to see those modules as an outer hexagon, the interfaces exposing module functionality being adapters to inner hexagon's ports.

The resulting hierarchy has been added.

Conclusion

Thus, Cryptomate was born. In the next post, we will start working on the main component of the system, that is the business logic and, most importantly, its ports.