Probabilistic data Structures – Bloom filter and HyperLogLog for Big Data

Oct 11

Probabilistic data Structures – Bloom filter and HyperLogLog for Big Data

When working with large volume of data memory and space requirement could be very high. This in turn have effect on scalability, when suddenly your job or process either taking too long or requires more resources. Probabilistic data structure allow you to trade some accuracy for immense decrease in memory usage. For example, with a single...

Read More

Streaming Platforms: Storm vs Spark Streaming

Sep 30

Streaming Platforms: Storm vs Spark Streaming

Our latest meetup, Storm vs Spark face-off, was a big hit among Big Data engineers in New York. Slides from our meetup and from Hadoop User Group in Chicago presented on this page. I hope both of those presentations will help you make better choice for your use case and environment. Apache storm vs. Spark Streaming P. Taylor Goetz, Apache...

Read More

Book Review – Big Data Analytics Beyond Hadoop: Real-Time Applications with Storm, Spark, and More Hadoop Alternatives

Sep 28

Book Review – Big Data Analytics Beyond Hadoop: Real-Time Applications with Storm, Spark, and More Hadoop Alternatives

“Big Data Analytics Beyond Hadoop: Real-Time Applications with Storm, Spark, and More Hadoop Alternatives” is a new book by Vijay Agneeswaran on the topic of Big Data. Author provides foundation why Hadoop, especially Map-Reduce computational model is not suited well for a number of cases. Author divided those cases into 3 broad...

Read More

Seven micro-services architecture problems and solutions

Jun 04

Seven micro-services architecture problems and solutions

Micro-service architecture present a set of challenges that need to be addressed. Those include: 1. Operational overhead. Now instead of single monolithic application you have to : Deploy many small micro-services Monitor many small micro services. Provision hardware for many more services. 2. Complexities in networking calls,...

Read More

Seven micro-services architecture advantages

Jun 03

Seven micro-services architecture advantages

Recently I had a conversation with an ops engineer on my team about why micro-service architecture makes a lot of sense and why we should embrace it. Most of our applications are monoliths built over years and having new, small micro-services is something new for our environment. “The term “Microservice Architecture” has...

Read More

The power of Groovy and Regular expressions

Dec 20

The power of Groovy and Regular expressions

Recently I came across Google Code Jam problem called Alien Language. The idea is to figure out based on a given pattern if work exist in the dictionary or not. Here is the task description from Google Code Jam: Problem After years of study, scientists at Google Labs have discovered an alien language transmitted from a faraway planet. The...

Read More

Meet Cornell NYC Tech

Sep 11

Meet Cornell NYC Tech

Today, with a group of other professionals from the industry, I was helping students at Cornell Tech NYC campus to practice real life job interviews. During my mock interviews and following conversation with students I learned quite a lot about Cornell’s Master of Engineering in Computer Science program. First of all, it’s...

Read More
Page 1 of 1012345...10...Last »