A Dino-mite Announcement! 🦖

Hey there smartx-ers,

The secret is out! Over the last few months, our Data Team has been working tirelessly to update our technology stack to make smartclip faster and more agile than ever before. On May 5, 2020, we managed to pull off one of the biggest releases in smartclip history – right under your nose! We replaced our proprietary data store with a new, Trino-based data storage solution that not only enhances the performance on all of our reporting applications but makes us self-sustainable and unstoppable 💪.

Say Goodbye to the Old Datastorus Rex 👋 🦖

Our old data storage solution was eight years old, which in technology terms, is literally a dinosaur. 

We’ve been wanting to make the switch for a while but needed to ensure that we didn’t sacrifice or lose any historical data. Well, after an arduous journey, we engineered a solution that keeps everything the way we built it for our users, but with more than enough room to grow in the future. Our new stack now consists of Trino as a query engine, SQL as a query language, Alluxio as an intermediate caching layer, and Parquet file format for the data storage.

Talking about data storage: In the last few months, we have stored more data than we have in all of smartclip history. So, when we say big data, we mean, like, really big data.

Moving forward, we now have the foundation we need to keep building upon our holistic tech stack, and now it will be easier to build on new technologies in the future.

The old and new systems have been running in parallel for nearly 3 months with no complaints. In fact, most of our colleagues did not even notice! You probably won’t even notice once we pull the plug on Old Datastorus Rex once and for all (or did we already do it? 🤷‍♂️).

Yours in product updates and dinosaur puns, 

smartclip Product Marketing Team