Existing financial data offerings are not built for modern technology. They can’t keep up with the scale and speed needed to support innovative financial applications. Smaller developers are priced out of the market due to solutions that are too expensive, restrictive, and complicated.
The financial industry has long lagged behind other markets when it comes to technology modernization. Pre-pandemic, only 55 percent of financial services organizations took steps to implement their digital transformation plans. The pandemic helped accelerate the process for many companies, but they still had to contend with legacy systems and inefficient operations.
IEX Cloud, an IEX Group company, tackled this problem with an innovative financial data delivery platform that makes quality financial data and services accessible to everyone.
The parent company, IEX Group, was founded in 2012 with a simple mission: build fairer markets. It started by creating a new stock exchange, called the Investors Exchange, focusing on transparency and access to data to level the playing field for long-term investors, such as mutual and pension funds, that represent the savings of millions of people.
IEX Cloud was founded in January 2019. Josh Blackburn, Head of Technology at IEX Cloud, said, “IEX Cloudʼs vision is to create a vibrant data ecosystem that delivers best in class data to the individual developers and the investor community, as well as those at Fintechs and enterprises large and small. The industry has long been held back by barriers that limit access to the data they need to innovate.”
Pioneering a Better Way to Build and Scale Financial Applications
IEX Cloud’s priorities were to:
- Break down these barriers and constantly reinvent to better serve the entire ecosystem.
- Challenge the status quo in the financial data industry by delivering high-quality data to their community that is easy to use and available at a reasonable price.
- Provide a powerful set of capabilities to help developers build efficiently and iterate quickly.
- Simplify legal, modernize access, and enable flexible pricing.
The platform provided powerful functionality for developers, with a flexible, scalable pricing model based on the data used. Customers are not locked into long-term contracts and substantial upfront payments. Instead, they get open access to a range of institutional-grade datasets with one subscription, split between core and premium data.
Core data includes:
- Real-time and historical stock prices
- Corporate actions
- Full U.S. market coverage
- Economic data
- Reference data
- IEX Investors Exchange data
- 95,000+ securities
- 20+ exchanges
- 100+ currency pairs
- And much more
Premium data comes from a growing community of curated partners, such as:
- Wall Street Horizon
- Fraud Factors
- Audit Analytics
- And much more
IEX Cloud was designed to be developer-friendly, making it as easy as possible to integrate data into their projects.
- All data is accessed through a single API. This API is based on REST, has resource-oriented URLs, returns JSON-encoded responses, and returns standard HTTP response codes.
- Support for C++, Go, .NET, R, Node.js, Python, Java, PHP, and Ruby.
- On-demand and streaming data availability.
- A no-code Rules Engine that automates serverless data analysis through custom event-driven alerts and data-driven conditions.
- Unlimited sandbox testing gives developers an environment to try free API calls and experiment with this financial data.
- The included cloud-based caching offers a fully managed service that automates data storage, scaling, high availability, monitoring, and administration.
- IEX Cloud provides a platform that can act as a developer’s backend, as some startups lack scalable infrastructure.
- Automated signup flow so users can get started immediately.
Data creators can use IEX Cloud as a turn-key distribution platform to distribute and commercialize their data. The platform takes care of the complete data delivery process by:
- Managing and scaling the API
- Tracking usage
- Billing customers
- Handling all other overhead
- Data normalization and symbology integration
These customers can integrate their data into IEX Cloud and gain instant access to a growing global user base or distribute to their pre-existing customers.
How IEX Cloud Customers Use the Platform
IEX Cloud’s flexibility and accessibility supported a wide range of use cases, such as:
- Front-office traders needing real-time data.
- Building out trade blotters and order management solutions.
- Understanding the market data and its risk in attribution implications on one’s portfolio.
- Supporting back-office operations and reconciliation.
- Keeping IR teams up to date about what’s going on in their company and near-competitors’ companies through news coverage, events, and streams of data delivered seamlessly through IEX Cloud’s API to their Business Intelligence (BI) solution of choice.
- Helping IR teams adjust legacy workflows to the work-from-home environment through a flexible platform.
Startups gain particular value from IEX Cloud. Dave Hannibal, Partnerships Manager at IEX Cloud, explained the platform’s advantage for this customer segment.
“We help them scale. They can come in and go from ideation through being a full-fledged fintech with millions of users.”
Hannibal continued, “We can work with them, and shepherd them all the way through that process, and scale with them, instantaneously. They never have to do a thing with us - as we continue to add more data on the platform, they can just grab and go. We are built in a way that helps them scale and then deliver on their value proposition. We have Fintechs all around the world who are building with us, who are integrating us, who are leveraging us for more and more data. It continues to grow across new startups and incumbents looking for new ways to redo old processes.”
Finding the Right Database to Help IEX Cloud Scale Beyond the Startup Phase
IEX Cloud had ambitious goals for its platform, but it needed to solve many technical challenges before it could offer the necessary speed and scale for modern financial applications. The company also had only three months to get the beta up and running.
Blackburn’s goal for the platform’s database was, “I wanted everything consolidated into one store, not a bunch of databases all over the place. Typically, you would split your data into different databases.”
He continued, “Amazon Redshift or Snowflake are great for storing large amounts of historical data and doing analytics. But neither database is going to meet a real-time service level agreement (SLA) for web developers, websites, or financial data. We want a delivery platform that scales to any number of users with real-time use cases, and any amount of data under the hood.”
IEX Cloud started out lean to get up and running with a few data sets. A managed MySQL database was selected initially because it was widely supported, quick to get running, and cost-effective for an early-stage startup.
The platform quickly outgrew that and ran into performance issues. As the team started adding more data and moving past the start-up phase, they needed something more web-scale.
Free Databases Are Not Free
The team chose Clickhouse to alleviate the immediate pain. It was a free solution that had high write speeds for overnight ETL processes.
However, free is not free. Clickhouse optimization was consuming a large percentage of the Site Reliability Engineers’ (SRE) time. Clustering didn’t work as advertised, and would fall over with too many concurrent reads. It was built for large analytical queries and few users.
IEX Cloud’s user base was growing, and Clickhouse wasn’t able to support millions of small reads. While it was great for time series data speed, it was too rigid. At this point, the team had data stored in MySQL, Clickhouse, and Redis.
Searching for Their Permanent Database Technology
After IEX Cloud’s first year and with revenue in the door, it was time to look for a database solution that would meet their needs and scale with the business. The team didn’t want to migrate databases again.
Financial data is complex, comes in many forms, and can change after the write. The database technology for IEX Cloud needed to be up for the challenge.
“We're trying to standardize any data set in the world, into one format and one API signature for our users. It's easier to learn. As we add data, it becomes readily available in the system,” explained Blackburn.