Building Freetrade

Live Pricing - how we built it

Tom Waszkowycz

August 11, 2021

Tom Waszkowycz

How we built live pricing.


When it comes to buying or selling an investment, knowing the price is pretty important. And while the Freetrade app has always been able to show you any price correct to the last few minutes, we knew we needed live pricing.

The good news is live pricing has arrived. And even better, I'm here to tell you how we did it. 


Why slow and steady doesn’t win this race

Dog Ride GIF

Our original pricing platform used pull-based polling, which fetches the latest stock prices from our price vendors on a fixed schedule, every few minutes. As an initial solution this was simple, easily maintainable and reliable.

But as it turned out, speeding this schedule up was not as straightforward. The changes we made struggled to scale as the number of stocks on our platform grew. And the unpredictability of the stock market caused yet another headache.


Share price updates
Charting the distribution of stock price updates.

Only a minority of our stocks are traded enough to see price changes every second. The majority change at a much lower frequency.

We still needed to capture this minority but as we increased the frequency (pushed that red line to the right), repeatedly polling the slower stocks meant wasted overheads rocketed.

Finding a more streamlined ride 

We can better fit this distribution by switching from a pull-based polling of every stock to push-based streaming, where we only receive a price update when it happens and avoid any wasted polling.

We didn’t have to look far to do this as our stock price vendors provided websocket and server-side event APIs - both offering much lower latency than a pull-based model can achieve.


The end. 


Well, not quite...


This is not a simple switch. Push-based connections need to be left running permanently so that we’re ready to receive any price update, at any time. 


The Freetrade platform is built on serverless Google Cloud Functions which only support short-lived operations so that the underlying resources can be managed flexibly. We needed to introduce a new environment to support these long running connections.


Finding the right environment

We investigated a few potential environments: Virtual machines, Google App Engine, and Kubernetes. 


We needed one that had a low knowledge barrier to entry, low maintenance cost, and one that was scalable and flexible enough to support future server-based use cases.

After much prototyping, testing, and discussions with both the Freetrade team and Google Engineers we decided on Google’s new managed Kubernetes platform: GKE Autopilot


GKE Autopilot provides the following features:

  • Little configuration is required as it provisions and manages the complexities usually introduced by Kuberenetes for us 
  • Automatic scaling to match our required compute capacity
  • Only charges for used compute resources (CPU, memory, storage) 
  • Follows security best practices 
  • Allows us to focus on solving other questions

So, we’re now ready to get price updates at a much better frequency! But how fast can we actually go?

Super Bowl Falling GIF by ADWEEK


Putting the stabilisers on 

Being fast is nice, but we need to make sure that these changes play nicely with the rest of our systems. 

There were two big concerns we needed to investigate:

1. Availability

Availability is a common problem for streaming applications: any downtime means that the pushed updates can’t be received, and that could mean missed prices!

To prevent this we use an approach built upon Kubernetes’ health probes to fail-fast and recover quickly. As soon as any problem is encountered we simply mark the streaming process as unhealthy. The health probes will spot this and automatically start up a new healthy pod for us with minimal downtime.

After a restart, data consistency is recovered by fetching the latest price for all stocks before we start streaming any new real time updates. This means that no matter how long the downtime, once the system is up and running the latest price data for any stock will be up to date.

2. Performance

Our performance testing used a mixture of running in sandbox environments and stress tests to check that the load can be handled by our stack and the usage wouldn’t break the bank.

The good news was our node.js Kubernetes pod could handle the load. But sending the data to our Firestore database was more of a bottleneck. In the end, we found that we could rely on BulkWriter to batch and queue updates for us, without introducing significant latency.

As a final precaution, we added a five second per stock throttle to guard our systems against any sudden spikes in price update frequency.

This means we’re ready for whatever the stock market throws at us. 

Drumroll...

Using our new pricing system you can now see live pricing for most stocks in the Freetrade app, updating up to every five seconds and with a latency of less than one second! 

As a customer, you can rely on fast and stable pricing as you travel on your investing journey.

Like A Boss Dog GIF


The road ahead 

The show goes on and there are many more improvements we’d like to make to our new pricing platform, including: 

  • Higher throughput by introducing more performant data stores
  • Improved scalability and reliability by sharding traffic across more Kubernetes pod instances
  • Use real time stock price updates to power new analytics for our users

Where you come in

Do you have the skills to test out new technologies, discover their limits, and use them to design reliable systems used by hundreds of thousands of users? 

If you want to help build a world where everyone is investing, come join the fun at freetrade.io/careers.

Freetrade careers

Important Information

When you invest, your capital is at risk. The value of your portfolio can go down as well as up and you may get back less than you invest. Past performance is not a reliable indicator of future results.

Freetrade is a trading name of Freetrade Limited, which is a member firm of the London Stock Exchange and is authorised and regulated by the Financial Conduct Authority. Registered in England and Wales (no. 09797821).

Sign up for our newsletter

Download the app and start
investing now.