Skip to content

Conversation

@JacksonBurns
Copy link
Contributor

Motivation or Problem

Resolves #2365

Description of Changes

Changes the GitHub CI to use micromamba instead of conda, which resolves the RMG environment in ~2 mins instead of 30+

Testing

Successfully ran on this fork.

Reviewer Tips

The fact that it runs is perhaps sufficient evidence. Might want to do a line-by-line comparison of how the depdencies were resolved to ensure that versions are consistent with conda.

The changes, if good, will also need to be ported into the documentation builder.

@JacksonBurns JacksonBurns marked this pull request as ready for review January 30, 2023 18:17
@JacksonBurns
Copy link
Contributor Author

JacksonBurns commented Jan 30, 2023

The debug level logging in the call to provision-micromamba should also be removed once the rest of this looks operational.

@codecov
Copy link

codecov bot commented Jan 30, 2023

Codecov Report

Merging #2366 (44bfa20) into main (a8c4127) will decrease coverage by 0.68%.
The diff coverage is n/a.

❗ Current head 44bfa20 differs from pull request most recent head aa97e23. Consider uploading reports for the commit aa97e23 to get more accurate results

@@            Coverage Diff             @@
##             main    #2366      +/-   ##
==========================================
- Coverage   48.25%   47.57%   -0.68%     
==========================================
  Files         110      110              
  Lines       30616    30612       -4     
  Branches     7982     7982              
==========================================
- Hits        14774    14564     -210     
- Misses      14298    14532     +234     
+ Partials     1544     1516      -28     
Impacted Files Coverage Δ
rmgpy/qm/molecule.py 37.55% <0.00%> (-43.27%) ⬇️
rmgpy/qm/mopac.py 26.43% <0.00%> (-41.38%) ⬇️
rmgpy/qm/qmdata.py 52.77% <0.00%> (-27.78%) ⬇️
rmgpy/qm/main.py 51.61% <0.00%> (-13.71%) ⬇️
rmgpy/rmg/reactors.py 19.95% <0.00%> (-1.04%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@JacksonBurns
Copy link
Contributor Author

I also propose that we run the CI on a schedule using GitHub actions scheduling. There have been a couple instances in the past couple weeks where the CI was failing on new PRs even when it was passing on main, only because it had not been run in months.

We could do it with this simple addition to the script:

on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron:  '0 0 8 ? * MON,TUE,WED,THU,FRI *'

which would run the CI every Monday-Friday at 8 am UTC (3 am EST)
Please let me know your thoughts @mjohnson541 @xiaoruiDong @kspieks

@JacksonBurns JacksonBurns requested a review from hwpang February 1, 2023 19:13
the micromamba configuration does not allow the version of python to be specified, so maybe we can set it up using a different action version. I suspect that incorrect python version is what caused the CI to fail to resolve the conda environment on the previous commit
the micromamba configuration does not allow the version of python to be specified, so maybe we can set it up using a different action version. I suspect that incorrect python version is what caused the CI to fail to resolve the conda environment on the previous commit
to find out about issues with greater frequency, this will force the CI to run on the main branch (where we are currently doing development) every weekday early in the morning
@JacksonBurns
Copy link
Contributor Author

I have added the mamba build to the documentation action, as well as the runner schedule to the CI action.

I did a comparison between the environment resolved by micromamba and conda (see it for yourself here and it resolves almost everything to be the same, with the exception of some operating system level packages that I assume are a function of the action being run on a different machine + VM at GitHub every time. All of our 'major' dependencies (those that we maintain or use heavily) are resolved to the correct major versions.

Tests are still passing and this cuts ~30 minutes off CI and Documentation build.

@JacksonBurns JacksonBurns added the Status: Ready for Review PR is complete and ready to be reviewed label Feb 5, 2023
@JacksonBurns JacksonBurns changed the title Ci with mamba Switch CI & Documentation Build to Micromamba from Conda for Reduced Execution Time Feb 5, 2023
channels: defaults,rmg,conda-forge
channel-priority: flexible
log-level: debug
log-level: warning
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we reset log level to warning here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you squash this with the commit that set it to debug?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug writes more to stdout than is useful for any future test running (see this run with debug) -- I only turned it on to deal with the issue of channel-priority. The default log level is warning

schedule:
# * is a special character in YAML so you have to quote this string
- cron: '0 0 8 ? * MON,TUE,WED,THU,FRI *'
- cron: '0 8 * * 1-5'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you squash this commit with the previous one?

Copy link
Contributor

@hwpang hwpang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Overall seems good with a few comments. Several of the commits can be squashed together. I check the fork, and it looks like it fails Trigger RMG-tests?

@JacksonBurns
Copy link
Contributor Author

Thanks for the PR! Overall seems good with a few comments. Several of the commits can be squashed together. I check the fork, and it looks like it fails Trigger RMG-tests?

I left a quick comment about the logging level. Will squash after it is resolved.

That action fails on forked repositories regardless of content. See here, where I changed nothing in the source code and it still fails at the same location.

JacksonBurns and others added 6 commits March 2, 2023 10:59
 - all depdencies which are package by others should be used instead of those packaged by us
 - small formatting change in github workflow file
libgfortran-ng was included in one of the packages but is now missing, add the conda-forge package
remove packages that appear to be unused, change those which can be drawn from conda-forge or other official changes where possible
@JacksonBurns
Copy link
Contributor Author

Moved to #2386

@JacksonBurns JacksonBurns deleted the CI-with-mamba branch March 2, 2023 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Ready for Review PR is complete and ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch CI + Documentation Build to use Mamba instead of Conda

2 participants