Kodi add-on testing
Ok, you develop a Kodi add-on and you would like to have CI for your code. What options do you have? How Kodi add-ons can be tested? And how to configure a CI for a Kodi add-on code? Let me share my experience.
Unit tests
According to the testing pyramid the bulk of your tests are unit tests:
The problem with Kodi is its python API is accessible only within Kodi runtime. For example, if you need to get the addon info you can use the following code:
import xbmcaddon
xbmcaddon.Addon().getAddonInfo("version")
xbmcaddon is a python interface for Kodi C++ code and it cannot be installed in your virtual
environment. You have to mock it. Combining python mocking library unittest.mock and
@RomanVM's Kodi stubs you can create some unit tests. But I must
admit that it might be not so trivial to write them. Your tests will be as good as your mocks are.
You might come to the situation when test maintenance takes more time than working on the add-on's
code. In my opinion, it makes sense to write unit tests for the code that doesn't use Kodi python
API. It might be various helpers, utilities, or really simple functions and classes that don't
require to mock a lot of side effects.
Of course, I highly recommend using pytest as a testing library. It's really the best tool for
test development. Moreover, it doesn't matter where your tests are located in the testing pyramid.
Integration tests
Integration tests assume that your code will be tested in the running Kodi instance. Now we need to solve the following problems to write the automation:
- we need a running Kodi and we should be able to specify its version;
- we should be able to install a development version of the testing add-on;
- we need to have an interface to interact with Kodi automatically;
- we should be able to check expected results.
Kodi setup
I have a strong opinion that containers ideally fit this task and other similar tasks such as
UI testing of web applications. In my previous post Conkodi
I explained how to create and run a container with Kodi. Using this container image we can configure
our setup stage in pytest's conftest.py:
import subprocess
import pytest
# it's just a helping function for podman CLI
def podman(*args):
subprocess.run(["podman"] + list(args), stdout=subprocess.DEVNULL)
# we want to have running Kodi container during whole testing session
@pytest.fixture(scope="session")
def run_kodi():
# remove containers from previous run
podman("rm", "-f", "kodi")
podman(
"run",
"--detach", # run a container in the background
"--name=kodi", # with the name "kodi"
# make various ports accessible for the host
"--publish=8080:8080", # Kodi JSON RPC listens to this port
"--publish=5999:5999", # this port needs accessing VNC server of the container
"--publish=9777:9777/udp", # Kodi EventServer listens to this port
"quay.io/quarck/conkodi:19", # the name and the tag of the container image with Kodi
)
yield
podman("stop", "kodi") # teardown, stop the container
Add-on installation
Kodi add-ons can be installed only via the UI. You need to find a zip archive in the filesystem and
click several buttons in Kodi. There is no CLI or API that allows the installation of an add-on.
What can we do here? First of all, let's make a small study of what happens when you install a Kodi
add-on. After the installation, the add-on archive is unpacked into addons directory. Besides, new
entries are created in the databases Addons33.db and Textures13.db. Knowing these facts we can
setup our container in such a way that Kodi will have the add-on installed. We can mount host
directories with the code of the add-on and the prepared database in a Kodi container. Let's update
the code from the previous paragraph:
import shutil
import subprocess
import pytest
def podman(*args):
subprocess.run(["podman"] + list(args), stdout=subprocess.DEVNULL)
@pytest.fixture(scope="session")
def build_plugin():
# Here we copy a development version of the add-on into the directory with where all Kodi
# add-ons are stored
shutil.copytree("/dir/with/addon/code", "/dir/with/addons/some.addon")
yield
shutil.rmtree("/dir/with/addons/some.addon")
@pytest.fixture(scope="session")
def run_kodi_container(build_plugin):
podman("rm", "-f", "kodi")
podman(
"run",
"--detach",
"--name=kodi",
"--publish=8080:8080",
"--publish=5999:5999",
"--publish=9777:9777/udp",
# Mounting a directory with a development version of the add-on
"--volume=/some/host/dir/addons/:/home/kodi/.kodi/addons",
# Mounting a directory with prepared databases
"--volume=/some/host/dir/Database/:/home/kodi/.kodi/userdata/Database",
"quay.io/quarck/conkodi:19",
)
yield
podman("pod", "stop", "kodi")
Kodi JSON RPC
Ok, we have a running container with Kodi and installed add-on and now it would be nice to have an
API for making various actions in Kodi. And there is some! Kodi provides JSON RPC API that allows
you to get a list of the items of a directory, execute addons with parameters, get properties of GUI
windows, and so on and so on. There are several python clients and I stopped my choice on
kodi-json. Virtually it supports all available methods. Let me demonstrate how it can be used:
from kodijson import Kodi
# Instantiate a Kodi object
kodi = Kodi("http://127.0.0.1:8080")
# Get list of items of the root directory of some addon
kodi.Files.GetDirectory(directory="plugin://some.addon")
We could even create a fixture in our test suite:
import pytest
from kodijson import Kodi
@pytest.fixture(scope="session")
def kodi(run_kodi_container):
return Kodi(JSON_RPC_URL)
And then use it in tests:
EXPECTED_ROOT_DIR = [
{
"file": "plugin://some.addon/some_item/",
"filetype": "file",
"label": "Some item",
"type": "unknown",
},
{
"file": "plugin://some.addon/some_dir/",
"filetype": "directory",
"label": "Some dir",
"type": "unknown",
},
]
def test_root_dir(kodi):
response = kodi.Files.GetDirectory(directory="plugin://some.addon")
assert ROOT_DIR == resp["result"]["files"]
That's the basics of the integration tests of a Kodi add-on. Despite JSON RPC doesn't allow you to do everything you can do via GUI in most cases it will be enough. You can even send keyboard button events and navigate through the UI but it might be error-prone.
References: