PLMBiasMeasureBenchmark

This repository contains the implementation and configs used for the papers

So can we use intrinsic bias measures or not? by Sarah Schröder, Alexander Schulz, Philip Kenneweg and Barbara Hammer (branch icpram23)
Semantic Properties of Cosine Based Bias Scores for Word Embeddings by Sarah Schröder, Alexander Schulz, Fabian Hinder and Barbara Hammer (see db_weat_examples.ipynb)
"The SAME score: Improved cosine based measure for semantic bias" (accepted at IJCNN 2024) by Sarah Schröder, Alexander Schulz and Barbara Hammer (branch ijcnn24)

See the respective branches or notebook to reproduce the experiments from those papers.

Requirements

All requirements are listed in requirements.txt.
The embeddings and embedding-bias-eval packages must be installed from source.

Download this repository, install the above mentioned requirements.

Name		Name	Last commit message	Last commit date
Latest commit History 136 Commits
configs		configs
data		data
utils		utils
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
bios.ipynb		bios.ipynb
bios_experiment.py		bios_experiment.py
crowspairs.ipynb		crowspairs.ipynb
db_weat_examples.ipynb		db_weat_examples.ipynb
icpram_eval.ipynb		icpram_eval.ipynb
init.sh		init.sh
initialize_experiment.py		initialize_experiment.py
jigsaw.ipynb		jigsaw.ipynb
jigsaw_experiment.py		jigsaw_experiment.py
mlm_experiment.py		mlm_experiment.py
multi_attr_bias_test.py		multi_attr_bias_test.py
new_unmasking_eval.py		new_unmasking_eval.py
requirements.txt		requirements.txt
robustness.ipynb		robustness.ipynb
robustness.py		robustness.py