Support only packaging specified requirements rather than the whole venv

[nickovs]

Currently "Zappa will automatically package your active virtual environment", which means that by default everything you are using for testing, linting, formatting and developing your code, and all the things that they depend upon, get included. While it is possible to exclude specific packages from the build, you need to know which ones are dependencies of your code and which are dependencies of the test environment. This tends to lead to package bloat.

It would be very helpful if Zappa supported providing a requirements file, and then only bundled up the packages listed in there and their dependencies. This would be in keeping with line two of the Zen of Python:

Explicit is better than implicit.

A generalised tool to resolve all dependencies is large and complex, but given the way that Zappa works at the moment, it doesn't need a generalised tool; all that is needed is a filter to limit which of the packages that are in the current virtual environment get packaged. For this all you need to do is parse the requirements (by calling packaging.requirements.Requirement on each meaningful line of the file), check the dependencies of the currently installed version (using importlib.metadata.requires(), checking the 'extra' tags), and repeat until you've done them all. This would yield a subset of the installed packages that is at least as version-consistent as the local venv and would not contain anything that has not explicitly been requested.

Would anyone else find this useful?

[github-actions]

Hi there! Unfortunately, this Issue has not seen any activity for at least 90 days. If the Issue is still relevant to the latest version of Zappa, please comment within the next 10 days if you wish to keep it open. Otherwise, it will be automatically closed.

[nickovs]

I think that this is still a good idea so I's posting to keep this open!

[monkut]

You are right, and I have re-built my local environment without dev requirements for deployment in the past.
I'm not sure if there is a clean way to derive what is a dev requirement and what is a production requirement.

Are you suggesting a ignore_packages type setting, to allow users to ignore packages explictly?

[nickovs]

Are you suggesting a ignore_packages type setting, to allow users to ignore packages explictly?

I think it would be much better to let the user provide a list of the packages they do want rather than a list of what they don’t want. If you use an ignore file then it requires the user to know every dependency of every package that you don’t or do want to include, and how they intersect. Also, people writing Python generally understand the concept and semantics of a requirements.txt file.

My suggestion would be to allow the user to specify a requirements file for deployment and Zappa would only package what’s listed there and their dependencies, and leave out anything not in that dependency tree.