Pipeline Pandits

Inspiration

Traditional asset management often suffers from a lack of real-time transparency. Our scalable app aims to build a client-first platform that fosters the trust between clients and the asset manager via increased real-time visibility.

What it does

We've developed a versatile wealth management system tailored for funds of all sizes, designed to revolutionise how wealth management firms access and utilise information. Our system features a streamlined pipeline delivering real-time price data on funds and equities. This includes a admin panel to customise the pipeline, a client facing app to view your investment funds and track them in real time, and a trader panel to get real time data and indicators on equities.

How we built it

We have utilised ReactJS for our frontend and we have used a robust and scalable backend utilising apache's Kafka and Spark. This means we can scale our system to 100s of thousands of users and stock prices. We have implemented a stream processing pipeline, meaning that as soon as data enters our system it is processed and sent to our users.

Challenges we ran into

Learning new technologies like the apache ones used for scalability. We tried experimenting with different stream processing pipelines, and had difficulty working with tools like Apache fling, and beam due to limited documentation and difficulty integrating into our Python pipeline. We had to eventually use Spark which we managed to get to work, and has been a really useful tool, as it is extremely scalable, and usable.

Furthermore we had integration issues with out Kafka consumers from the Node.js side, however once we got it up and running, we were seamlessly able to stream data onto our frontend.