Integration of `Hamilton` DAG flow for feature engineering

[Nikronic]

Intro

As you can guess from the notebook file and also vizard.data.preprocessor that the whole feature engineering part is a mess! Hamilton library seems like a valid solution.

Note: If using this library leads to resolving the aforementioned challenges, I think it's going to be our go-to for solving any project involving feature engineering.

Description

To fix the mess, a cleaner approach that is systematic and does not require "looking and running cell-by-cell" is desired. It seems (I am not sure otherwise I would have made it a HIGH PRIORITY to get it done asap) that Hamilton library does cover all of our challenges, including:

• tracking dependencies of columns on each other: sometimes feature C is produced by first modifying A and B (i.e., B,C -> A)
• testing cell by cell
• easier integration of code and feature engineering inside the library (i.e., vizard)
• lightweight and human-readable

Note: Currently I assign myself as the only assignee until @aliinreallife let me know if he wants to join.

---

pre-requirements:

• does @aliinreallife want to join this?
• does this library actually solve the problem? (requires discussing and some minor tests before directly using on the code base)

[aliinreallife]

@Nikronic of course I would love that and would definitely take a good look at the subject i think we can discuss it very soon

[Nikronic]

@aliinreallife you don't to check/respond when you are not at work! It's totally fine!
Nonetheless, thank you for letting me know.

[aliinreallife]

@Nikronic thanks but since I'm in the subway i have some free time to check git and respond to some of the massages