Posts

Showing posts from February, 2017

Programming with Apache Spark and Cassandra -draft

Putting the knowledge gained so far in this  and frequent questions that many may ask and what we have asked ourselves. 1.1          What is the need for using Spark ? Spark gives you horizontal scale ability in a programmer friendly way. 1.2          But what about other options ? There are other options as well. I have listed them below, which describes and highlights Sparks place in the architecture Type Level of Granularity Descritpion LoadBalancer (nginx,haproxy) Request Level (usually HTTP requests) Works well for Request-response type client server protocols. Works also well in context of microservices in application program side However to scale the processing insdie the application programs this is inadequate Task Managers (celery, other MQ based) Task Level Helps to scale processing in the application program.Takes care of Task handling. However the onus is on the developer