Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 25 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
[![code style: runic](https://img.shields.io/badge/code_style-%E1%9A%B1%E1%9A%A2%E1%9A%BE%E1%9B%81%E1%9A%B2-black)](https://github.com/fredrikekre/Runic.jl)


This package implements special types of vectors and associated methods for hyperdimensional computing. Hyperdimensional computing (HDC) is a paragdigm to represent patterns by means of a high-dimensional vectors (typically 10,000 dimensions). Specific operations can be used to create new vectors by combining the information or encoding some kind of position. HDC is an alternative machine learning method that is extremely computationally efficient. It is inspired by the distributed, holographic representation of patterns in the brain. Typically, the high-dimensionality is more important than the nature of the operations. This package provides various types of vectors (binary, graded, bipolar...) with sensible operations for *aggragating*, *binding* and *permutation*. Basic functionality for fitting a k-NN like classifier is also supported.
This package implements special types of vectors and associated methods for hyperdimensional computing. Hyperdimensional computing (HDC) is a paragdigm to represent patterns by means of a high-dimensional vectors (typically 10,000 dimensions). Specific operations can be used to create new vectors by combining the information or encoding some kind of position. HDC is an alternative machine learning method that is extremely computationally efficient. It is inspired by the distributed, holographic representation of patterns in the brain. Typically, the high-dimensionality is more important than the nature of the operations. This package provides various types of vectors (binary, graded, bipolar...) with sensible operations for *aggragating*, *binding* and *permutation*.

We provide a set of types of hypervectors (HVs), with the associated operations.

## Basic use

Expand All @@ -15,41 +17,44 @@ Several types of vectors are implemented. Random vectors can be initialized of d
```julia
using HyperdimensionalComputing

x = BipolarHDV() # default length is 10,000
x = BipolarHV() # default length is 10,000

y = BinaryHDV(20) # different length
y = BinaryHV(20) # different length

z = RealHDV(Float32) # specify data type
z = RealHV(Float32) # specify data type
```

The basic operations are `aggregate` (creating a vector that is similar to the provided vectors), `bind` (creating a vector that is dissimilar to the vectors) and `circshift` (shifting the vector inplace to create a new vector). For `aggregate` and `bind`, we overload `+` and `*` as binary operators, while `Π` is an alias for `circshift`. The latter is lazily implemented. All functions have an inplace version, using the `!` prefix.
The basic operations are `bundle` (creating a vector that is similar to the provided vectors), `bind` (creating a vector that is dissimilar to the vectors) and `circshift` (shifting the vector inplace to create a new vector). For `bundle` and `bind`, we overload `+` and `*` as binary operators, while `ρ` is an alias for `shift`.

```julia
x, y, z = GradedHDV(10), GradedHDV(10), GradedHDV(10)
x, y, z = GradedHV(10), GradedHV(10), GradedHV(10)

# aggregation

aggregate([x, y, z])
bundle([x, y, z])

x + y
x + y # binary operator for bundling

# binding

bind([x, y, z])
x * y
bind(x, y)

x * y # binary operator for binding

# permutation

circshift(x, 2) # shifts the coordinates
Π(x, 2) # same
shift(x, 2) # circular shifts the coordinates
ρ(x, 2) # same

Π!(y, 1) # inplace
ρ!(y, 2) # inplace
```

See the table for which operations are used for which type.
See the table below for which operations are used for which type.

## Embedding sequences

TODO: update!

HDC is particularly powerful for embedding sequences. This is done by creating embeddings for n-grams and aggregating the n-grams found in the sequence.

```julia
Expand All @@ -69,32 +74,13 @@ threegrams = compute_3_grams(basis)
sequence_embedding(sequence, threegrams)
```

## Training

A model is basically trained by making an aggregation of all elements within a class. Training is simple. Prediction is done by nearest-neighbor search based on `similarity`.

```julia
hdvs = [BipolarHDV() for i in 1:1000] # 1000 vectors
y = rand(Bool, 1000) # two labels

centers = train(y, hdvs)

predict(BipolarHDV(), centers) # predict for a random vector
predict(hdvs, centers) # repredict the labels
```

In practice, this leads to suboptimal performance. One can retrain the model by reaggregating the wrongly classified labels.

```julia
retrain!(centers, y, hdvs, niters=10)
```

## Overview of operations

| Vector | element domain | aggregate | binding | similarity |
| ------ | --------------| ---------| ----------| --------|
| `BinaryHDV` | 0, 1 | majority | xor | Jaccard |
| `BipolarHDV` | -1, 0, 1 | sum and threshold | multiply | cosine |
| `GradedHDV` | [0, 1] | 3π | fuzzy xor | Jaccard |
| `GradedBipolarHDV` | [-1, 1] | 3π | fuzzy xor | cosine |
| `RealHDV` | real | sum weighted to keep vector norm | multiply | cosine |
| `BinaryHV` | {0, 1} | majority | xor | Jaccard |
| `BipolarHV` | {-1, 1} | sum and threshold | multiply | cosine |
| `TernaryHV` | {-1, 0, 1} | sum and threshold | multiply | cosine |
| `GradedHV` | [0, 1] | 3π | fuzzy xor | Jaccard |
| `GradedBipolarHV` | [-1, 1] | 3π | fuzzy xor | cosine |
| `RealHV` | real (normally distributed) | sum weighted to keep vector norm | multiply | cosine |
1 change: 1 addition & 0 deletions src/HyperdimensionalComputing.jl
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ using Distances, Random, Distributions, LinearAlgebra

export AbstractHV, BinaryHV, BipolarHV,
GradedBipolarHV, RealHV, GradedHV, TernaryHV
export normalize!, normalize
export bundle, bind, shift!, shift, ρ, ρ!, perturbate, perturbate!
export sequence_embedding, sequence_embedding!
export compute_1_grams, compute_2_grams, compute_3_grams, compute_4_grams, compute_5_grams,
Expand Down
Loading
Loading