Debug Test Executorlib

[jan-janssen]

No description provided.

[github-actions]

👈 Launch a binder notebook on branch pyiron/pyiron_workflow/exedebug

[jan-janssen]

@liamhuber 1.8.1 works fine - I am going to start updating from here.

[jan-janssen]

exception calling callback for <Future at 0x7fc5b5e25820 state=finished returned int>
Traceback (most recent call last):
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/pyiron_workflow/mixin/run.py", line 353, in _finish_run
    raise e
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/pyiron_workflow/mixin/run.py", line 347, in _finish_run
    unique_executor.shutdown(wait=False)
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/executorlib/executor/base.py", line 162, in shutdown
    self._task_scheduler.shutdown(wait=wait, cancel_futures=cancel_futures)
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/site-packages/executorlib/task_scheduler/base.py", line 205, in shutdown
    self._process.join()
  File "/home/runner/work/pyiron_workflow/pyiron_workflow/cached-miniforge/my-env/lib/python3.12/threading.py", line 1146, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

[liamhuber]

Awesome, thanks @jan-janssen! If I'm reading that sed correctly, the patch in pyiron/executorlib#943 makes the tests pass here again

[liamhuber]

Hmm, the coverage failure is a legitimate failure, not just a hiccup uploading.

This line is causing a real problem, but at least the entire stack trace is inside pwf

[jan-janssen]

@liamhuber How can I include this ugly patch in the coverage test?

[liamhuber]

That's a good question. I see it and I'm looking around

[liamhuber]

Here is where I mixin and override standard SlurmClusterExecutor behaviour...

[liamhuber]

Perhaps in that class we could introduce an override to shutdown?

def shutdown(...):
    if ...:
        special new behaviour
    else:
        super().shutdown(...)

[liamhuber]

It would be good to try something like that, as monkeypatching CacheOverride with the fix that will exist in future versions of BaseExecutor might resolve the coverage integration tests at the same time as the slurm tests...

[liamhuber]

Stacking a PR to give it a try #813

[liamhuber]

Hmm, the coverage failure is a legitimate failure, not just a hiccup uploading.

This line is causing a real problem, but at least the entire stack trace is inside pwf

Just to quickly note, it's only failing for the dummy test class:

pyiron_workflow/pyiron_workflow/executors/wrapped_executorlib.py

Line 79 in 7035b23

class _CacheTestClusterExecutor(CacheOverride, TestClusterExecutor): ...

[jan-janssen]

@liamhuber Just to write down what I understood so far. The add_done_callback() is executed when I cancel the future object, so we see the corresponding error message. The reason I do this is that the future object is no longer updated once the executor is closed, so I would like to stay with the new behavior but I have to think about how we could integrated it with pyiron_workflow.

[liamhuber]

@liamhuber Just to write down what I understood so far. The add_done_callback() is executed when I cancel the future object, so we see the corresponding error message. The reason I do this is that the future object is no longer updated once the executor is closed, so I would like to stay with the new behavior but I have to think about how we could integrated it with pyiron_workflow.

That is fair. Is there any way of filtering the done callbacks to only trigger on certain types of "done-ness", i.e. in pyiron_workflow we attach the future in a way that it won't trigger if the future is done because of a force-cancel?

[liamhuber]

Is there any way of filtering the done callbacks to only trigger on certain types of "done-ness", i.e. in pyiron_workflow we attach the future in a way that it won't trigger if the future is done because of a force-cancel?

No.

If the future has already completed or been cancelled, fn will be called immediately.

[jan-janssen]

@liamhuber I just wanted to validate the tests work with 1.9.0 in addition to 1.8.2

[jan-janssen]

With the patch the tests work fine - we are just not able to directly apply it to Test-and-Coverage test. So I am going to close this pull request in favor of the solution in #814