Llm rag bot

[Zubair Maalick (zmaalick)]

Added a basic solution with some TODO notes.

It crawls the cylc8 documentation and loads it (not all pages at a moment) and create two different vector store (Chroma and Faiss), will also create with pinecone. Idea is to test which one is more efficient. There are some other TODO notes in notebook, that are mainly related to computing. On VDI, I could do this much. For further improvements, Azure resources are needed.

[mo-glenngreed]

This work was noticed by the impact it caused upon the network and firewalls related to the Azure SPICE/vdi system. This had knock on impacts on other users. I request that this work is not repeated on the vdi and it finds a separate resource. We will investigate why a user can impact others in this way and look for mitigations. But until we are comfortable we can mitigate this from reoccurring we request that this is not something the DS CoP pursues on the vdi.