EmbeddingBiasScores

Implementations and wrapper of bias scores for text embeddings.

Requirements

All requirements are listed in requirements.txt.
In the notebooks, we use a wrapper for huggingface models. You can install the lib from source:
https://github.com/UBI-AGML-NLP/Embeddings

Installation

Download this repository and install via pip install .

Example

A minimalistic example for the usage of implemented bias scores can be found in example.ipynb.

Implemented Scores

So far, we implemented geometrical bias scores that measure the association of embeddings with protected groups. Other scores and bias test will follow.
Papers that introduced the scores are linked.

Geometrical Bias scores

• SAME
• WEAT
• generalized WEAT
• Direct Bias
• RIPA
• MAC

Clustering based Bias Scores

Clustering, Classification and Neighbor Test