When I started working at Red Hat my first job was contribution to integration tests of one web
application. The tests are written Python and the repository has both
requirements.txt. But why?
Python and many other interpreted languages such as Ruby and JS require a prepared environment to
run some code. This environment should have all dependencies installed with correct versions. In
Python we use
requirements.txt file for describing such environment. It's just a text file with
package names and versions. e.g.:
pip we can recreate this environment in any other place. It makes a lot of sense when
we deploy a Python application.
But what about dependencies of dependencies? Python code is distributed via packages. A Python
package is a directory that follows a certain file structure and has
setup.py file that is a
build script for
setuptools. If your package uses other packages you can specify them in
import setuptools with open("README.md", "r", encoding="utf-8") as fh: long_description = fh.read() setuptools.setup( name="example-pkg-YOUR-USERNAME-HERE", # Replace with your own username version="0.0.1", author="Example Author", author_email="email@example.com", description="A small example package", long_description=long_description, long_description_content_type="text/markdown", url="https://github.com/pypa/sampleproject", install_requires=["pytest>6.0.0", "django"] packages=setuptools.find_packages(), classifiers=[ "Programming Language :: Python :: 3", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", ], python_requires='>=3.6', )
Applications vs packages
So we now know packages need only
setup.py in order to specify dependencies.
Application and standalone scripts need only
requirements.txt for reproducing an environment.
Things get more complicated when your package is an application as well.
setuptools can generate
a python script that can be placed in
$PATH. It can call some entry point function in your
package. During the deployment we would like to have a reproducible environment and as I told
requirements.txt is what we need to use to recreate such environments.
Let me consider the case that I started the post with.
pytest is the best choice for developing
tests on any level: unit, functional and integration. The integration tests I worked on are really
huge. In order to manage the complexity we created some abstractions to interact with the web
application we want to test. These abstractions are organized as a package. In the tests we import
that package and in fact we test its code. Let me provide a simplified directory structure:
mypkg/ __init__.py app.py view.py tests/ test_app.py test_view.py setup.py requirements.txt
Interesting thing here is that our tests is the application :) And for deploying our application
in CI we need to recreate the environment and that's why we have both
requirements.txt in one repository. Of course nothing prevents you to divide the tests and the
package in different repositories but I think it would be inconvenient.