Skip to content

Commit 91056f4

Browse files
LiamPattinsonLiamPattinson
authored
Merge pull request #6 from ResearchCodingClub/ci_edits
Overhaul CI lesson
2 parents 81b7a79 + 875f3cc commit 91056f4

7 files changed

Lines changed: 319 additions & 79 deletions

File tree

‎episodes/10-CI.Rmd‎

Lines changed: 247 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: "Continuous Integration with GitHub Actions"
3-
teaching: 10
4-
exercises: 2
3+
teaching: 20
4+
exercises: 25
55
---
66

77
:::::::::::::::::::::::::::::::::::::: questions
@@ -20,39 +20,61 @@ exercises: 2
2020

2121
## Continuous Integration
2222

23-
Continuous Integration (CI) is the practice of automating the merging of code changes into a project.
24-
In the context of software testing, CI is the practice of running tests on every code change to ensure that the code is working as expected.
25-
GitHub provides a feature called GitHub Actions that allows you to integrate this into your projects.
23+
Continuous Integration (CI) is the practice of automating the merging of code
24+
changes into a project. In the context of software testing, CI is the practice
25+
of running tests on every code change to ensure that the code is working as
26+
expected. GitHub provides a feature called GitHub Actions that allows you to
27+
integrate this into your projects.
2628

27-
In this lesson we will go over the very basics of how to set up a GitHub Action to run tests on your code.
29+
In this lesson we will go over the basics of how to set up a GitHub Action
30+
to run tests on your code.
31+
32+
:::::: prereq
33+
34+
This lesson assumes a working knowledge of Git and GitHub. If you get stuck,
35+
you may find it helpful to review the Research Coding Course's
36+
[material on version control](https://researchcodingclub.github.io/course/#version-control-introduction-to-git-and-github)
37+
38+
:::::::::::::
2839

2940
## Setting up your project repository
3041

31-
- Create a new repository on GitHub for this lesson called "python-testing-course" (whatever you like really)
32-
- Clone the repository into your local machine using `git clone <repository-url>` or GitKraken if you use that.
42+
- Create a new repository on GitHub for this lesson called
43+
"python-testing-course" (whatever you like really). We
44+
recommended making it public for now.
45+
- Clone the repository into your local machine using `git clone
46+
<repository-url>` or via Github Desktop.
3347
- Move over all your code from the previous lessons into this repository.
3448
- Commit the changes using `git add .` and `git commit -m "Add all the project code"`
35-
- Create a new file called `requirements.txt` in the root of your repository and add the following contents:
49+
- Create a new file called `requirements.txt` in the root of your repository
50+
and add the following contents:
3651

3752
```
3853
pytest
3954
numpy
40-
pandas
41-
pytest-mpl
42-
pytest-regtest
43-
matplotlib
55+
snaptol
4456
```
4557

46-
This is just a list of all the packages that your project uses and will be needed later.
47-
Recall that each of these are used in various lessons in this course.
58+
This is just a list of all the packages that your project uses and will be
59+
needed later. Recall that each of these are used in various lessons in this
60+
course.
4861

62+
:::::: callout
63+
64+
Nowadays it is usually preferable to list dependencies in a file called
65+
`pyproject.toml`, which also allows Python packages to be installed and
66+
published. Look out for our upcoming course on reproducible environments to
67+
learn more!
68+
69+
::::::::::::::
4970

5071
Now we have a repository with all our code in it online on GitHub.
5172

5273
## Creating a GitHub Action
5374

54-
GitHub Actions are defined in `yaml` files (these are just simple text files that contain a list of instructions). They are stored
55-
in the `.github/workflows` directory in your repository.
75+
GitHub Actions are defined in `yaml` files -- a structured text file which is
76+
commonly used to pass settings to programs. They are stored in the
77+
`.github/workflows` directory in your repository.
5678

5779
- Create a new directory in your repository called `.github`
5880
- Inside the `.github` directory, create a new directory called `workflows`
@@ -66,93 +88,239 @@ Let's add some instructions to the `tests.yaml` file:
6688
# This is just the name of the action, you can call it whatever you like.
6789
name: Tests (pytest)
6890

69-
# This is the event that triggers the action. In this case, we are telling GitHub to run the tests whenever a pull request is made to the main branch.
91+
# This sets the events that trigger the action. In this case, we are telling
92+
# GitHub to run the tests whenever a push is made to the repository.
93+
# The trailing colon is intentional!
7094
on:
71-
pull_request:
72-
branches:
73-
- main
95+
push:
7496

75-
# This is a list of jobs that the action will run. In this case, we have only one job called build.
97+
# This is a list of jobs that the action will run. In this case, we have only
98+
# one job called test.
7699
jobs:
77-
build:
78-
# This is the environment that the job will run on. In this case, we are using the latest version of Ubuntu, however you can ues other operating systems like Windows or MacOS if you like!
79-
runs-on: ubuntu-latest
80-
81-
# This is a list of steps that the job will run. Each step is a command that will be executed on the environment.
82-
steps:
83-
# This command tells GitHub to use a pre-built action. In this case, we are using the actions/checkout action to check out the repository. This just means that GitHub will use this repository's code to run the tests.
84-
- uses: actions/checkout@v3 # Check out the repository on github
85-
# This is the name of the step. This is just a label that will be displayed in the GitHub UI.
86-
- name: Set up Python 3.10
87-
# This command tells GitHub to use a pre-built action. In this case, we are using the actions/setup-python action to set up Python 3.10.
88-
uses: actions/setup-python@v3
89-
with:
90-
python-version: "3.10"
91-
92-
# This step installs the dependencies for the project such as pytest, numpy, pandas, etc using the requirements.txt file we created earlier.
93-
- name: Install dependencies
94-
run: |
95-
python -m pip install --upgrade pip
96-
pip install -r requirements.txt
97-
98-
# This step runs the tests using the pytest command. Remember to use the --mpl and --regtest flags to run the tests that use matplotlib and pytest-regtest.
99-
- name: Run tests
100-
run: |
101-
pytest --mpl --regtest
100+
101+
# This is the name of the job
102+
test:
103+
104+
# This is the environment that the job will run on. In this case, we are
105+
# using the latest version of Ubuntu, however you can use other operating
106+
# systems like Windows or MacOS if you like!
107+
runs-on: ubuntu-latest
108+
109+
# This is a list of steps that the job will run. Each step is a command
110+
# that will be executed on the environment.
111+
steps:
112+
113+
# This command tells GitHub to use a pre-built action. In this case, we
114+
# are using the actions/checkout action to check out the repository. This
115+
# just means that GitHub will clone this repository to the current
116+
# working directory.
117+
- uses: actions/checkout@v6
118+
119+
# This is the name of the step. This is just a label that will be
120+
# displayed in the GitHub UI.
121+
- name: Set up Python 3.12
122+
# This command tells GitHub to use a pre-built action. In this case, we
123+
# are using the actions/setup-python action to set up Python 3.12.
124+
uses: actions/setup-python@v6
125+
with:
126+
python-version: "3.12"
127+
128+
# This step installs the dependencies for the project such as pytest,
129+
# numpy, pandas, etc using the requirements.txt file we created earlier.
130+
- name: Install dependencies
131+
run: |
132+
python -m pip install --upgrade pip
133+
pip install -r requirements.txt
134+
135+
# This step runs the tests using the pytest command.
136+
- name: Run tests
137+
run: |
138+
pytest
102139
```
103140
104-
This is a simple GitHub Action that runs the tests for your code whenever a pull request is made to the main branch.
141+
This is a simple GitHub Action that runs the tests for your code whenever code
142+
is pushed to the repository, regardless of what was changed in the repository
143+
or which branch you push too. We'll see later how to run tests only when
144+
certain criteria are fulfilled.
105145
106146
## Upload the workflow to GitHub
107147
108148
Now that you have created the `tests.yaml` file, you need to upload it to GitHub.
109149

110150
- Commit the changes using `git add .` and `git commit -m "Add GitHub Action to run tests"`
111-
- Push the changes to GitHub using `git push origin main`
151+
- Push the changes to GitHub using `git push`
152+
153+
This should trigger a workflow on the repository. While it's running, you'll see an orange
154+
circle next to your profile name at the top of the repo. When it's done, it'll change to
155+
a green tick if it finished successfully, or a red cross if it didn't.
156+
157+
![GitHub repository view with a green tick indicating a successful workflow run](fig/github_repo_view.png){alt="GitHub repository view with a green tick indicating a successful workflow run"}
158+
159+
You can view all previous workflow runs by clicking the 'Actions' button on the
160+
top bar of your repository.
161+
162+
![GitHub Actions button](fig/github_actions_button.png){alt="GitHub Actions Button"}
163+
164+
If you click on the orange circle/green tick/red cross, you can also view the
165+
individual stages of the workflow and inspect the terminal output.
166+
167+
![Detailed view of a GitHub workflow run](fig/github_action.png){alt="Detailed view of a GitHub workflow run"}
168+
169+
170+
## Testing across multiple platforms
171+
172+
A very useful feature of GitHub Actions is the ability to test over a wider
173+
range of platforms than just your own machine:
174+
175+
- Operating systems
176+
- Python versions
177+
- Compiler versions (for those writing C/C++/Fortran/etc)
178+
179+
We can achieve this by setting `jobs.<job_id>.strategy.matrix` in our workflow:
180+
181+
```yaml
182+
jobs:
183+
test:
184+
strategy:
185+
matrix:
186+
python_version: ["3.12", "3.13", "3.14"]
187+
os: ["ubuntu-latest", "windows-latest"]
188+
runs-on: ${{ matrix.os }}
189+
steps:
190+
...
191+
```
192+
193+
Later in the file, the `setup-python` step should be changed to:
194+
195+
```yaml
196+
- name: Set up Python ${{ matrix.python_version }}
197+
uses: actions/setup-python@v6
198+
with:
199+
python-version: ${{ matrix.python_version }}
200+
```
201+
202+
By default, all combinations in the matrix will be run in separate jobs. The
203+
syntax `${{ matrix.x }}` inserts the text from the `x` list for the given matrix job.
204+
205+
::::::::::::::::::::::::::::::::::::: challenge
206+
207+
## Upgrade the workflow to run across multiple platforms
208+
209+
- Make the changes above to your workflow file, being careful to get the indentation right!
210+
- Commit the changes and push to GitHub.
211+
- Check the latest jobs in the Actions panel.
212+
213+
:::::::::::::::::::::::: solution
112214

113-
## Enable running the tests on a Pull Request
215+
You should see that a total of 6 jobs have run, and hopefully all will have passed!
114216

115-
The typical use-case for a CI system is to run the tests whenever a pull request is made to the main branch to add a feature.
217+
![Image showing completed matrix jobs.](fig/matrix_tests.png){alt="Completed matrix tests."}
116218

117-
<!-- figure
118-
![branch protection visual instruction](episodes/fig/github-branch-protection.png) -->
119-
- Go to your GitHub repository
120-
- Click on the "Settings" tab
121-
- Scroll down to "Branches"
122-
- Under "Branch protection rules" / "Branch name pattern" type "main"
123-
- Select the checkbox for "Require status checks to pass before merging"
124-
- Select the checkbox for "Require branches to be up to date before merging"
219+
:::::::::::::::::::::::::::::::::
220+
221+
::::::::::::::::::::::::::::::::::::::::::::::::
125222

126-
This makes it so when a Pull Request is made, trying to merge code into main, it will need to have all of its tests passing
127-
before the code can be merged.
128223

129-
Let's test it out.
224+
This ensures that code that runs on your machine should, in theory, run on many
225+
other peoples' machines too. However, it's best to restrict the matrix to the
226+
minimum number of necessary platforms to ensure you don't waste resources. You
227+
can do so with a list of exclusions:
130228

131-
- Create a new branch in your repository called `subtract` using `git checkout -b subtract`
132-
- Add a new function in your `calculator.py` file that subtracts two numbers, but make it wrong on purpose:
229+
```yaml
230+
strategy:
231+
matrix:
232+
python_version: ["3.12", "3.13", "3.14"]
233+
os: ["ubuntu-latest", "windows-latest"]
234+
# Only run windows on latest Python version
235+
exclude:
236+
- os: "windows-latest"
237+
python_version: "3.12"
238+
- os: "windows-latest"
239+
python_version: "3.13"
240+
````
241+
242+
## Running on other events
243+
244+
You may have wondered why there is a trailing colon when we specify `push:` at
245+
the top of the file. The reason is that we can optionally set additional
246+
conditions on when CI jobs will run. For example:
133247

134-
```python
135-
def subtract(a, b):
136-
return a + b
248+
```yaml
249+
on:
250+
push:
251+
# Only check when Python files are changed.
252+
# Don't need to check when the README is updated!
253+
paths:
254+
- '**.py'
255+
- 'pyproject.toml'
256+
# Only check when somebody raises a push to main.
257+
# (not recommended in general!)
258+
branches: [main]
137259
```
138260

139-
- Then add a test for this function in your `test_calculator.py` file:
261+
Doing this can prevent pointless CI jobs from running and save resources.
262+
263+
You can also run on events other than a push. For example:
140264

141-
```python
142-
def test_subtract():
143-
assert subtract(5, 3) == 2
265+
```yaml
266+
on:
267+
push:
268+
paths:
269+
- '**.py'
270+
- 'pyproject.toml'
271+
# Run on code in pull requests.
272+
pull_request:
273+
paths:
274+
- '**.py'
275+
- 'pyproject.toml'
276+
# This allows you to launch the job manually
277+
workflow_dispatch:
144278
```
145279

146-
- Commit the changes using `git add .` and `git commit -m "Add subtract function"`
147-
- Push the changes to GitHub using `git push origin subtract`
280+
There is an important subtlety to running on `pull_request` versus
281+
`push`:
282+
283+
- `push` runs directly on the commits you push to GitHub.
284+
- `pull_request` runs on the code that would result _after_ the pull request
285+
has been merged into its target branch.
286+
287+
In collaborative coding projects, it is entirely possible that `main` will have
288+
diverged from your branch while you were working on it, and tests that pass on
289+
your branch will fail after the merge. For this reason, it's recommended to
290+
always include both `push` and `pull_request` in your testing workflows.
291+
292+
::::::::::::::::::::::::::::::::::::: challenge
293+
294+
## Running on pull requests (advanced)
148295

149-
- Now go to your GitHub repository and create a new Pull Request to merge the `subtract` branch into `main`
296+
Can you engineer a situation where a CI job passes on `push` but
297+
fails on `pull_request`?
150298

151-
You should see that the GitHub Action runs the tests and fails because the test for the `subtract` function is failing.
299+
- Write a new function, commit the changes, and push it to your `main`
300+
branch.
301+
- Switch to a new branch `my_branch` with `git switch -c my_branch`,
302+
and write a test for that function.
303+
- Check that the test passes, and commit it.
304+
- Push `my_branch` to GitHub with `git push -u origin my_branch`,
305+
but don't raise a pull request yet.
306+
- Return to your `main` branch, and modify the function being tested.
307+
- Push the changes to `main`.
308+
- Now raise a pull request from `my_branch` into `main`.
309+
310+
:::::::::::::::::::::::: solution
311+
312+
The code on the new branch will be testing the old implementation,
313+
and should pass. However, following the merge, the test would fail.
314+
This results in the `push` test passing, and the `pull_request` test
315+
failing.
316+
317+
![Example of tests failing on pull requests.](fig/pull_request_test_failed.png){alt="Example of tests failing on pull requests."}
318+
319+
:::::::::::::::::::::::::::::::::
320+
321+
::::::::::::::::::::::::::::::::::::::::::::::::
152322

153-
- Let's now fix the test and commit the changes: `git add .` and `git commit -m "Fix subtract function"`
154-
- Push the changes to GitHub using `git push origin subtract` again
155-
- Go back to the Pull Request on GitHub and you should see that the tests are now passing and you can merge the code into the main branch.
323+
## Keypoints
156324

157325
So now, when you or your team want to make a feature or just update the code, the workflow is as follows:
158326

@@ -171,7 +339,7 @@ This will greatly improve the quality of your code and make it easier to collabo
171339
- Continuous Integration (CI) is the practice of automating the merging of code changes into a project.
172340
- GitHub Actions is a feature of GitHub that allows you to automate the testing of your code.
173341
- GitHub Actions are defined in `yaml` files and are stored in the `.github/workflows` directory in your repository.
174-
- You can use GitHub Actions to only allow code to be merged into the main branch if the tests pass.
342+
- You can use GitHub Actions to ensure your tests pass before merging new code into your `main` branch.
175343

176344
::::::::::::::::::::::::::::::::::::::::::::::::
177345

‎episodes/fig/github_action.png‎

95.1 KB
Loading

‎episodes/fig/github_actions_button.png‎

8.88 KB
Loading

‎episodes/fig/github_repo_view.png‎

34.3 KB
Loading

‎episodes/fig/matrix_tests.png‎

29.7 KB
Loading

‎episodes/fig/pull_request_test_failed.png‎

49.2 KB
Loading

0 commit comments

Comments
 (0)