Skip to content

Make it obvious when a BUG is caused by a user's choice of plugin #2081

@ssssam

Description

@ssssam

BuildStream is infinitely extensible using plugins written in Python. That's really great, but problems in plugins lead to broken builds, and it can give the impression that BuildStream itself is unreliable.

As an example, consider the log output from endlessm/eos-build-meta#163 (edited for brevity):

  gnome-build-meta.bst:gnomeos-deps/zram-generator.bst:
    [00:00:01][][   fetch:gnome-build-meta.bst:gnomeos-deps/zram-generator.bst] BUG     Fetch

        An unhandled exception occured:
    
        Traceback (most recent call last):
          File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/jobs/job.py", line 350, in child_action
            result = self.child_process()  # pylint: disable=assignment-from-no-return
                     ^^^^^^^^^^^^^^^^^^^^
          File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/jobs/elementjob.py", line 81, in child_process
            return self._action_cb(self._element)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/queues/fetchqueue.py", line 75, in _fetch_not_original
            element._fetch(fetch_original=False)
    ...
          File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/source.py", line 1508, in _stage
            self.stage(directory)
          File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/git_repo.py", line 125, in stage
            mirror.stage(directory)
          File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/_git_utils.py", line 313, in stage
            objects = collect_objects(mirror, commit, base, True)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/_git_utils.py", line 73, in collect_objects
            entry.commit for entry in mirror.get_walker(commit.id, excluded)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        TypeError: BaseRepo.get_walker() takes from 1 to 2 positional arguments but 3 were given

To the casual reader, this log says: there is a BUG, somewhere in BuildStream's Python code.

With training in how to read the log, you might figure out the nuance: there is a bug somewhere in buildstream-plugins-community.

And if you are one of the elite group of people who have worked on buildstream-plugins-community before, you might see this and realize: this is a problem caused by the dulwich python library.

If you are in the 99.9% of people who don't know much about BuildStream's internals, it simply looks BuildStream is badly coded and it crashes at random on some machines --- which is really unfortunate because BuildStream aims to achieve quite the opposite, and make builds repeatable on different machines.

Since anyone can write a plugin, there is no way to avoid plugins failing some of the time. What is missing is a way to clearly identify which plugin caused the failure.

Implementing this probably requires some extra code around every point that BuildStream calls into a plugin, which should be non-trivial but doable.

I think it's better for everyone if the log output in this kind of case looks more like:


    [00:00:01][][   fetch:gnome-build-meta.bst:gnomeos-deps/zram-generator.bst] PLUGIN ERROR     Fetch (using `git_repo` source plugin)

        An unhandled exception occured in the `git_repo` source plugin:
    
        Traceback (most recent call last):
            ....
        TypeError: BaseRepo.get_walker() takes from 1 to 2 positional arguments but 3 were given

       This is not a problem with BuildStream's core -- report it to the developers of the plugin.
       The `git_repo` source plugin was provided by junction element: `freedesktop-sdk.bst:plugins/buildstream-plugins.bst`.

The intention is to make it clear that BuildStream's core is working as expected, and the user needs to consider their choice of plugins.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions