v0.1.0

Added

• Added an end-to-end implementation of the metric

v0.0.9

Fixed

• Added edlib to setup.py and requirements.txt.

v0.0.8

Added

• Added trying to fix the predicted character offsets using an alignment algorithm
• Added an option to return the QA result as a dict

Changed

• Refactored the Scorer code to be cleaner

Fixed

• Specifying the spacy version to 2.2.4 in setup.py

v0.0.7

Added

• Added returning approximate character offsets in the context for the QA model's prediction

v0.0.6

Added

• Added "silent" options for the question generation and answering models

0.0.5

Adding LERC metric to QAEval; Fixing broken 0.0.4 release.

Fixing breaking transformers change

I believe the config file for facebook/bart-large changed, it was a breaking change for the generation model that was forced. See this issue. This tagged commit fixes the incompatibility.

Adding urllib requirement

In the demo colab notebook, directly pip installing qaeval works, but importing the library does not because of some error caused by the default urllib package version. It works with urllib>=1.25.10, so we include that as a requirement in this release.

Arxiv release

Releasing the code for the arxiv submission