Tag: Data Engineering

Introduction In the realm of finance and analytics, having access to macroeconomic data that is both accurate and actual is of the highest importance for making informed decisions. At Narhwal Data Solutions we have been working closely with a client to transform their data integration process. We are proud to report that we have been […]

Don’t expect magic from dbt and don’t expect it will fix all your problems. Instead, expect to get a stable framework that makes your project as simple as possible. The only side effect of simplification I noticed is that I face much fewer problems and the ones I encounter are usually simple to debug.

If you’re data enthusiast, it’s definitely worth trying. Dbt (data build tool) is relatively fresh (v. 1.2) and open-source tool that perfectly fills the gap in data engineering stack. It’s dedicated for data engineers that love to code and hate to drag-and-drop. Particularly, for analytics engineers that work somewhere between data engineers and data analysts

There is no single definition of what the data lake is. Moreover, there have been many misconceptions around the idea. To understand the concept we move back to its origins.

The concept of a data warehouse is quite old, it originates from late 1980’s. Yet, in our practice we find out the term “data warehouse” is often misunderstood. The presence of a similar term, a data lake, does not makes things easier and the two concepts are frequently confused or mixed.

In the other post on Coronavirus reporting we touched briefly the topic of dimensions. In particular we discussed about the temporal dimension. Since dimensional modelling is a basic concept in the data warehouse design it deserves a separate post. The temporal dimension is commonly used in practically every business and science domain. Time / date is commonly considered as an […]