-
Notifications
You must be signed in to change notification settings - Fork 39
Description
BuildStream is infinitely extensible using plugins written in Python. That's really great, but problems in plugins lead to broken builds, and it can give the impression that BuildStream itself is unreliable.
As an example, consider the log output from endlessm/eos-build-meta#163 (edited for brevity):
gnome-build-meta.bst:gnomeos-deps/zram-generator.bst:
[00:00:01][][ fetch:gnome-build-meta.bst:gnomeos-deps/zram-generator.bst] BUG Fetch
An unhandled exception occured:
Traceback (most recent call last):
File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/jobs/job.py", line 350, in child_action
result = self.child_process() # pylint: disable=assignment-from-no-return
^^^^^^^^^^^^^^^^^^^^
File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/jobs/elementjob.py", line 81, in child_process
return self._action_cb(self._element)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/queues/fetchqueue.py", line 75, in _fetch_not_original
element._fetch(fetch_original=False)
...
File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/source.py", line 1508, in _stage
self.stage(directory)
File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/git_repo.py", line 125, in stage
mirror.stage(directory)
File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/_git_utils.py", line 313, in stage
objects = collect_objects(mirror, commit, base, True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/_git_utils.py", line 73, in collect_objects
entry.commit for entry in mirror.get_walker(commit.id, excluded)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: BaseRepo.get_walker() takes from 1 to 2 positional arguments but 3 were given
To the casual reader, this log says: there is a BUG, somewhere in BuildStream's Python code.
With training in how to read the log, you might figure out the nuance: there is a bug somewhere in buildstream-plugins-community.
And if you are one of the elite group of people who have worked on buildstream-plugins-community before, you might see this and realize: this is a problem caused by the dulwich python library.
If you are in the 99.9% of people who don't know much about BuildStream's internals, it simply looks BuildStream is badly coded and it crashes at random on some machines --- which is really unfortunate because BuildStream aims to achieve quite the opposite, and make builds repeatable on different machines.
Since anyone can write a plugin, there is no way to avoid plugins failing some of the time. What is missing is a way to clearly identify which plugin caused the failure.
Implementing this probably requires some extra code around every point that BuildStream calls into a plugin, which should be non-trivial but doable.
I think it's better for everyone if the log output in this kind of case looks more like:
[00:00:01][][ fetch:gnome-build-meta.bst:gnomeos-deps/zram-generator.bst] PLUGIN ERROR Fetch (using `git_repo` source plugin)
An unhandled exception occured in the `git_repo` source plugin:
Traceback (most recent call last):
....
TypeError: BaseRepo.get_walker() takes from 1 to 2 positional arguments but 3 were given
This is not a problem with BuildStream's core -- report it to the developers of the plugin.
The `git_repo` source plugin was provided by junction element: `freedesktop-sdk.bst:plugins/buildstream-plugins.bst`.
The intention is to make it clear that BuildStream's core is working as expected, and the user needs to consider their choice of plugins.