CurryProxy

A fast and performant proxy and aggregator for querying multiple instances of an API spread across globally distributed data centers.

Capabilities

• CurryProxy can handle multiple routes for use against different revisions of an API or for use against completely unrelated APIs. See etc/routes.sample.yaml for an example configuration.
• Simple request forwarding (useful for preserving backwards compatability with pre-CurryProxy versions of an API hosted in a single datacenter)
  • Example: GET https://api.example.com/v1.0/foo/bar forwarded to GET https://1.api.example.com/v1.0/foo/bar
• Advanced request forwarding to multiple endpoints
  • Example: GET https://api.example.com/1,2/v1.0/foo/bar forwarded to the following:
    • GET https://1.api.example.com/v1.0/foo/bar
    • GET https://2.api.example.com/v1.0/foo/bar
  • Requests are made in parallel
  • JSON responses received from multiple endpoints are aggregated and returned to the client
    • Example: {"foo": 1} received from 1 and {"bar": 2} received from 2 are aggregated to [{"foo": 1}, {"bar": 2}] and returned to the client
  • Rich, meaningful errors logged and returned to the client when a proxied request fails

Installation

pip install curryproxy will do what you expect.

There are a couple of extra options:

• pip install curryproxy[tests]. This will install any packages required to run the tests.
• pip install curryproxy[frozen]. This will install curryproxy with explicitly pinned dependencies, as generated by pip freeze. This is intended as a convenience for those who want the package to provide a reproducible environment. The versions used will be the latest working ones at the time the given Curry version was released.

You will need to provide configuration files before starting Curry.

NOTE: The configuration files have changed in v2.x. routes.json and logging.conf have become routes.yaml and logging.yaml respectively. See the sample files for examples.

Limitations

• Merging responses from multiple endpoints together occurs only under certain conditions outlined on the Multiple Endpoints Aggregation wiki page.

Testing

Tests are set up to run with tox.

The default tox setup creates environments for each known-usable version of each of our dependencies, and runs the tests in each of them. That amounts to around a dozen environments at the moment, so you'll probably want to run them in parallel with tox -p auto.

If you don't want to be that thorough (perhaps you're smoke testing a work in progress), you can run against only the most recent supported versions of dependencies with tox -e py36.

You can test against specific dependency versions like this:

tox -e py36-requests2.{2,3,4,5,6,7} -p auto
tox -e py36-webob1.{2,3,4,5} -p auto
tox -e py36-grequests0.{2,3} -p auto
tox -e py36-requests.latest

tox -e lint is also available, and will run various linting checks.

Roadmap

• OData support for server-side ordering and paging
• Response caching for a specified time