Dialect map I/O

About

This repository contains the Input / Output capabilities to be used in data-ingestion jobs.

It will be used as a Python dependency in a small variety of data-ingestion pipelines (static-data, NLP metrics...), in order to send curated information to the Dialect Map database, making use of the private Dialect map API.

Dependencies

Python dependencies are specified within the pyproject.toml file.

In order to install the development packages, as long as the defined commit hooks:

pip install ".[all]"
pre-commit install

Formatting

All Python files are formatted using Black, and the custom properties defined in the pyproject.toml file.

make check

Testing

Project testing is performed using Pytest. In order to run the tests:

make test

In addition to the common unit-tests, there are specific groups of tests that require non Python tools to be installed first. Those tests have been marked using Pytest so that they are not run by default.

The complete list of markers is defined within the pyproject.toml file.

To run a particular group of tests, set the TESTS_MARKERS environment variable to the name of the desirable marked group to run, or set it empty for full execution. Example:

export TESTS_MARKERS=""
make test

Tagging

Commits can be tagged to create informal releases of the package. In order to do so:

1. Bump up the package version (VERSION) following Semantic Versioning.
2. Add a new section to the CHANGELOG.
3. Create and push a tag: make tag.