How Big things change the rules

What basketball teaches us about IoT databases

When Lew Alcindor arrived at UCLA in 1965, he already had a winning pedigree. After losing only 6 games in his four high school years (out of more than 100 played) and winning more than 70 consecutive, he wasn’t the average college freshman. Add to that his height (7'2") and athletic ability, and UCLA had a reason to be excited. That being said, the university had the #1 ranked team in the country, and had won the championship the last two years in a row, so for Alcindor to stand out, he’d have to be extra special.

Due to NCAA regulations at the time, Alcindor was prohibited from playing on the varsity team when he arrived and instead, led the freshman team. But when the freshman team, midway through the year, was given the opportunity to play the varsity team in an exhibition game, the Alcindor lead freshmen beat the varsity team by 15 points — yes, the freshman team beat the national champions by 15 points. Alcindor was extra special, and he was just getting started.

By the time his sophomore year arrived, Alcindor was ready to show the world the full breadth of his talents. In the 66–67 season, he averaged 29 points a game, and the Bruins won all 30 of their games leading them to a National Championship in the NCAA tournament, in which Alcindor was named tournament MVP.

Unfortunately, he may have dominated a little too much in his sophomore year. It should come as no surprise that Alcindor’s height presented a significant advantage on the court, as it allowed him to dunk with ease. So much so that the NCAA thought it gave him an unfair advantage — he was literally too big for the game — so in the spring of ’67, they changed their rules, banning dunking from the league (a rule which stood for almost a decade).

Fortunately for Alcindor, when the rules changed he was able to change too, and when the NCAA prevented him from dunking, he developed a vicious hook shot (known as the skyhook) which was even harder to defend than his dunk. At the end of his college career, he’d won a total of 88 game and only lost 2 (one of which he’d played with an eye injury), captured 3 National Championships, and was name tournament MVP three times (something that’s never been repeated).

He then went onto the NBA, where he changed his name to Kareem Abdul-Jabbar, and proceeded to win 7 NBA titles and to this day is widely regarded as the best center to ever play the game.

For the first 40+ years of its existence, the database world has followed a certain set of rules. Systems were designed either for fast transactions, or fast analytics, but not both, data was persisted locally (on the compute node) for low latency of both of writes and reads, and almost invariably, as data sizes grew, compute was required to grow in step. The arrival of the massive amounts of data in IoT workloads over the past decade however, is forcing all of those rules to change.

IoT data is dominating the world. According to IDC, the rate of global data growth will exceed 61% annually for the next 6 years, resulting in an additional 142 zetabytes of data being created. The bulk of this data will be machine generated IoT data. To put 142 zetabytes into perspective, the largest hard disk available for purchase today is 16 terabytes. It would take nearly 9 billion of these enormous drives to contain 142 zetabytes. It’s a truly remarkable amount of data.

This tremendous data growth — which is too big for traditional database systems — requires a rethinking of the traditional database rules and a redesign of existing systems so that they can efficiently store and query IoT data.

Specifically, IoT databases must:

The rules have changed. Only a database which addresses these newly identified design constraints will excel in the IoT world.

Recently, IBM launched the latest version of Db2 Event Store, a data store designed specifically for IoT workloads in the following ways:

Additionally, in the 2.0 release, Db2 Event Store has been integrated with the Db2 Common SQL Engine, the most sophisticated SQL-based analytics query engine available. Regardless of your query needs Db2 Event Store can handle them quickly and efficiently.

If you’re looking for a solution to your IoT problems, we have them covered. Feel free to reach out for more information.

Adam has been developing and designing complex software systems for the last two decades. He is also a son, brother, husband and father.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store