Skip to content

Commit 08b88e8

Browse files
Knowledge entry on how to create custom providers in faker for factory boy (#28)
* Explaining how to setup a Github Actions workflow for running django tests * Explaining how to develop a custom provider for Faker to be used in factory_boy * Fixing bare urls * typo --------- Co-authored-by: Carlos Martinez <c.martinez@runtime-revolution.com>
1 parent 4f0146e commit 08b88e8

File tree

2 files changed

+193
-8
lines changed

2 files changed

+193
-8
lines changed

docs/testing/6_factory_boy.md

Lines changed: 64 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Factory boy
22

3-
https://factoryboy.readthedocs.io/en/stable/index.html
3+
<https://factoryboy.readthedocs.io/en/stable/index.html>
44

55
As a fixtures replacement tool, it aims to replace static, hard to maintain fixtures with easy-to-use factories for
66
complex objects.
@@ -26,10 +26,12 @@ class Band(models.Model):
2626
custom_id = models.TextField(unique=True)
2727
```
2828

29+
---
30+
2931
## Faker
3032

31-
- factory boy with faker https://factoryboy.readthedocs.io/en/stable/reference.html#faker
32-
- faker https://faker.readthedocs.io/en/latest/
33+
- factory boy with faker <https://factoryboy.readthedocs.io/en/stable/reference.html#faker>
34+
- faker <https://faker.readthedocs.io/en/latest/>
3335

3436
Factory boy has many ways to generate data, usually we prefer to use Faker,
3537
factory boy provides a wrapper for faker
@@ -45,13 +47,13 @@ class MusicTrackFactory(factory.django.DjangoModelFactory):
4547
```
4648

4749
Basically this is calling the `name` function in Faker library
48-
https://faker.readthedocs.io/en/latest/providers/faker.providers.person.html#faker.providers.person.Provider.name
50+
<https://faker.readthedocs.io/en/latest/providers/faker.providers.person.html#faker.providers.person.Provider.name>
4951

5052
### Faker attribute with params
5153

5254
Let's say we want our music durations to be between 30 and 500 seconds
5355
we can use `random_int` like in
54-
https://faker.readthedocs.io/en/latest/providers/baseprovider.html#faker.providers.BaseProvider.random_int
56+
<https://faker.readthedocs.io/en/latest/providers/baseprovider.html#faker.providers.BaseProvider.random_int>
5557

5658
:x: **Common Mistake:**
5759

@@ -76,11 +78,59 @@ class MusicTrackFactory(factory.django.DjangoModelFactory):
7678

7779
### Other providers
7880

79-
Here is a list of other fakers classes https://faker.readthedocs.io/en/latest/providers.html
81+
You can find more providers [here](https://faker.readthedocs.io/en/latest/providers.html). If you can't find the provider you need, you can also check [here](https://faker.readthedocs.io/en/latest/communityproviders.html) for community maintained providers.
82+
83+
If you still can't find the provider you need. You can always develop your own custom provider.
84+
85+
### Custom providers
86+
87+
You can very easily create your own custom Faker providers to supplement your factories with better, more relevant data. For example, let's say you want better names for the MusicTrack model instead of generating a person's name.
88+
89+
```python
90+
import factory
91+
from faker.providers import BaseProvider
92+
93+
class MusicTrackNameProvider(BaseProvider):
94+
__provider__ = "track_name"
95+
song_names_1 = [
96+
"The Awakening",
97+
"The Voyagers",
98+
"The Empyreal",
99+
"Of Carnage and",
100+
"Callisto",
101+
"The Scourge",
102+
"The Thirteen",
103+
]
104+
song_names_2 = [
105+
"of the Stars",
106+
"Beneath the Mare Imbrium",
107+
"Lexicon",
108+
"a Gathering of the Wolves",
109+
"Rising",
110+
"of the Fourth Celestial Host",
111+
"Cryptical Prophecies of Mu",
112+
]
113+
114+
def track_name(self):
115+
return f"{self.random_element(self.track_names_1)} {self.random_element(self.track_names_2)}"
116+
117+
# This is the important bit that allows you to simply invoke the provider with the name "track_name".
118+
factory.Faker.add_provider(MusicTrackNameProvider)
119+
120+
class MusicTrackFactory(factory.django.DjangoModelFactory):
121+
# Invoking the provider is as simple as calling the value defined in __provider__.
122+
name = factory.Faker("track_name")
123+
```
124+
125+
As you can see, this is very simple to do and you can easily make your factories output data with higher quality. You can be as creative and versatile with the data that is generated as you need. For this example, it simply combines random values from two different lists.
126+
127+
For code-organization reasons, you can always implement your providers in a providers.py file and have your factories in the factories.py file.
128+
129+
---
80130

81131
## Relations and Foreign Keys
82132

83-
https://factoryboy.readthedocs.io/en/stable/recipes.html#dependent-objects-foreignkey
133+
<https://factoryboy.readthedocs.io/en/stable/recipes.html#dependent-objects-foreignkey>
84134

85135
You can use SubFactory to create other dependent models like:
86136

@@ -93,9 +143,11 @@ class MusicTrackFactory(factory.django.DjangoModelFactory):
93143
Band = factory.SubFactory(BandFactory)
94144
```
95145

146+
---
147+
96148
## Unique constraints
97149

98-
https://factoryboy.readthedocs.io/en/stable/orms.html#factory.django.DjangoOptions.django_get_or_create
150+
<https://factoryboy.readthedocs.io/en/stable/orms.html#factory.django.DjangoOptions.django_get_or_create>
99151

100152
for django we use django_get_or_create, for unique fields so we don't have exception when running tests
101153

@@ -117,6 +169,8 @@ class MusicTrackFactory(factory.django.DjangoModelFactory):
117169
django_get_or_create = ("name","band",)
118170
```
119171

172+
---
173+
120174
## Final Factories
121175

122176
```python
@@ -138,6 +192,8 @@ class MusicTrackFactory(factory.django.DjangoModelFactory):
138192
django_get_or_create = ("name","band",)
139193
```
140194

195+
---
196+
141197
## If conditions within Factory (factory.Maybe Method)
142198

143199
Let's consider the following Factory example:

docs/testing/7_github_actions.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# Github Actions
2+
3+
[Documentation](https://docs.github.com/en/actions)
4+
5+
Github Actions is a feature that allows you to automate and execute different processes related to the development of your repository.
6+
7+
For example, you can always run all tests locally before you commit any changes to a repository or you can automate this process by using Github Actions.
8+
9+
Read this [page](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions) carefully as it explains the basics of how Github Actions works.
10+
11+
## Example for a Django project with Poetry
12+
13+
The following code covers how to setup a workflow in Github Actions that will run your tests.
14+
It makes some assumptions, like the python version that it uses, poetry for package dependency and postgres for the database. However, you can infer what may be necessary according to your environment setup, as the workflow is neatly divided into steps.
15+
16+
### Setup
17+
18+
The basic setup is creating a .github folder in the root of your repository, then a workflows folder inside the .github folder, and then a django.yml file inside the workflows folder. Folder structure summary:
19+
20+
- .github
21+
- workflows
22+
- django.yml
23+
24+
### The Workflow
25+
26+
The workflow is defined in the django.yml file.
27+
28+
```yaml
29+
# The visible name that will appear in the github interface.
30+
name: Django Tests
31+
32+
# You can specify branches and other actions, but the on: push directive is one of the most basics. Everytime there's a push on any branch, this workflow will be triggered and will execute its jobs.
33+
on: push
34+
35+
# You can define environment variables on a global level for the workflow.
36+
# Here we define variables for the postgres database.
37+
# More information on how environment variables work here: https://docs.github.com/en/actions/learn-github-actions/variables#defining-environment-variables-for-a-single-workflow
38+
env:
39+
POSTGRES_DB: postgres
40+
POSTGRES_USER: postgres
41+
POSTGRES_PASSWORD: postgres
42+
POSTGRES_HOST: 127.0.0.1
43+
POSTGRES_PORT: 15433
44+
45+
jobs:
46+
# Defines the name of the first and only job in this workflow.
47+
django_tests:
48+
# This job will run on a Ubuntu Linux runner with the Ubuntu version 22.04.
49+
# Because this job requires a postgres database and it uses a docker container for that purpose. The Runner must be a Linux-based OS.
50+
runs-on: ubuntu-22.04
51+
52+
# Services are Docker containers. For this job, we are using a postgres container for the database.
53+
# See more information regarding the services here: https://docs.github.com/en/actions/using-containerized-services/about-service-containers
54+
services:
55+
# A name for the service.
56+
postgres:
57+
# Use postgres version 15.
58+
image: "postgres:15"
59+
# Define the variables for the postgres container.
60+
env:
61+
POSTGRES_PASSWORD: ${{env.POSTGRES_PASSWORD}}
62+
POSTGRES_USER: ${{env.POSTGRES_USER}}
63+
POSTGRES_DB: ${{env.POSTGRES_DB}}
64+
# Note that we didn't use ${{env.POSTGRES_PORT}} in the ports section. That's because you don't have access to the env object in the ports section.
65+
# Read more about this here: https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability
66+
# TODO: If anyone knows if there's a way to use the env value in the ports, feel free to make a PR.
67+
ports:
68+
# Here we are saying that the port 5432 is exposed outwards through port 15433.
69+
- 15433:5432
70+
# These are just some optional yet recommended health checks.
71+
options: >-
72+
--health-cmd pg_isready
73+
--health-interval 10s
74+
--health-timeout 5s
75+
--health-retries 5
76+
77+
# steps is important as it divides a job into smaller parts.
78+
# In this case, the job is to run django tests, and we can divide that job into smaller steps.
79+
steps:
80+
# This is the first step. The "uses" keyword specifies that this step will run v3 of the actions/checkout action. This is an action that checks out your repository onto the runner, allowing you to run scripts or other actions against your code (such as build and test tools).
81+
# You can read more about this action here: https://github.com/actions/checkout#readme
82+
- uses: actions/checkout@v3
83+
84+
# You can also name your steps so that everything is clearer.
85+
- name: Set up Python 3.11
86+
# This setup-python action will do exactly what it suggests, install python in your runner's OS. Note that v4 is not the version of python, it's the version of the action.
87+
uses: actions/setup-python@v4
88+
with:
89+
python-version: "3.11"
90+
91+
# I chose to do a custom installation of poetry, but there are actions that will handle this for you. For example: https://github.com/snok/install-poetry
92+
- name: Install and configure Poetry
93+
run: |
94+
INSTALL_PATH="$HOME/.local"
95+
INSTALLATION_SCRIPT="$(mktemp)"
96+
VIRTUALENVS_PATH="{cache-dir}/virtualenvs/#\~/$HOME"
97+
98+
curl -sSL https://install.python-poetry.org/ --output "$INSTALLATION_SCRIPT"
99+
100+
POETRY_HOME=$INSTALL_PATH python3 "$INSTALLATION_SCRIPT" --yes --version="1.3.2"
101+
102+
export PATH="/root/.local/bin:$PATH"
103+
104+
poetry config virtualenvs.create true
105+
poetry config virtualenvs.in-project true
106+
poetry config virtualenvs.path "$VIRTUALENVS_PATH"
107+
108+
echo "VENV=.venv/bin/activate" >> "$GITHUB_ENV"
109+
110+
# The last step installed and configured poetry, now we can install our dependencies.
111+
- name: Install Dependencies
112+
run: |
113+
export PATH="/root/.local/bin:$PATH"
114+
poetry install --no-interaction
115+
116+
# This is all very self-explanatory, everything is configured and ready for execution, simply invoke the tests.
117+
- name: Run Tests
118+
run: |
119+
source $VENV
120+
cd project_name && python manage.py test
121+
```
122+
123+
### Results
124+
125+
Once you've commit-pushed your workflow, you can access the actions page of your repository. For example, this knowledge base, also has a continuous integration workflow defined, you can the actions page [here](https://github.com/runtimerevolution/python-knowledge-base/actions).
126+
127+
You can inspect each run individually and see if it's running correctly or not, or if the tests are accusing something wrong.
128+
129+
If there is something wrong, like an ill-defined configuration or an error in a step, you have to fix and add-commit-push, which is laborious but a necessary evil. To work around this, you can use something like [act](https://github.com/nektos/act). However, these tools aren't perfect and for example, act doesn't work with the services containers, so this workflow would never work in act; but it can be used for smaller and simple workflows because you can test locally without having to constantly commit-push every change you want to test.

0 commit comments

Comments
 (0)