The Big data day at Indic Threads Delhi 2013 started with session by Gagan Aggarwal on "Using Graph Databases for Insights into Connected Data" He took us through the basics of Graph databases, how they are fundamentally different from R-DBMS and NoSQL stores and then show cased some advanced features which can be useful in certain application A relevant blog post covering a use case on above topic can be looked at http://xebee.xebia.in/index.php/2013/08/24/neo4j-vs-rdbms-handling-access-control-in-document-management-system/ development scenarios.

First session gave a new dimension to the audience who got to know database in a different way that was earlier known to them and also made a good start for the day to continue with next session by Srihari Srinivasan on "SQL on Hadoop - Is SQL the next big thing for Hadoop". Srihari took the audience through following types of query processing methods that Hadoop can support

i) Querying using Hive Query Language

ii) Distributed Querying on Hadoop in which he talked about Bloom filters

iii) Split Query Processing by Hadoop

iv) Cloudera Impala to issue low-latency SQL queries to data stored in HDFS

The talk also started a day long question about the relevance of Big Data / Hadoop hype, is it really useful and why should an organization have Hadoop at first place, to which Srihari told us that it is based on the problem or the requirement that one has which will call for using it or discarding it.

Previous insightful session was followed by much more in to the code session by tech-geek and Apache Crunch committer Rahul Sharma who gave a session on "Building Hadoop pipelines using Apache Crunch".Rahul also mentioned the advantages of this approach which not only helped in faster development but also in cleaner approach as writing Unit Test cases with this approach covers the limitations of other unit testing frameworks like MRUnit. Manoj Mohan gave a high energy filled session on "Big Data Search Simplified with Elastic Search". Harpreet Singh gave a very insightful session on "Building an Enterprise Big Data Platform for 100TB Dataset". He shared with us case studies on the projects he had worked on. He shared his experience in big data. To give a complete architectural view for a Big Data solution, he told that a Big Data solution is a combination of cloud infrastructure and other components.

The overall experience of attending IndicThreads was quite enriching in terms of the knowledge gained.