Data Engineering

Data Engineering involves the design, development, and management of data infrastructure and pipelines. It focuses on collecting, processing, transforming, and storing data in a scalable and efficient manner. Data Engineers build systems that enable data analytics, machine learning, and other data-driven applications, ensuring reliable and timely access to high-quality data for insights and decision-making.

Big Data

Rise of Elastic Data Warehouse and Database Services

Currently the majority of cloud based database and data warehouse services are provisioned with fixed storage and compute resources. Resizing of resources cannot be performed without compromising availability and performance. This means service users typically end up with over-provisioned under-utilised expensive resources to accommodate possible...

Big Data

Requirements for stream processing architecture

In 2005 Stonebraker et al. published a paper that outlined 8 key requirements for stream processing architecture. These key requirements can be easily translated into building blocks of stream processing architecture. Although, this article dates before systems such as Apache Kafka, Amazon Kinesis, Apache Spark,...

Subscribe to Abhishek Tiwari

Subscribe to our newsletter and receive access to exclusive posts and content updates. We don't spam and your email won't be shared with third-parties.

Thanks for subscribing!

Check your inbox to confirm your subscription.

Please enter a valid email address!