I am currently working as the lead data science infrastructure developer. I have architected & built Big Data Analytics solution for 2TB+ data repository from scratch Key Roles & Responsibilities : 1. Designed and implemented end-to-end solution to process/explore/analyse event logs 2. Working with teams to define KPIs, product on-boarding funnels and user segmentation 3. Developed Spark sql programs to compute aggregations, trends & funnels iteratively 4. Successfully executed Spark environment tuning & Debugged sql query joins to gain performance 5.Optimised data Re-partitioning with respect to cluster nodes to ensure complete resource utilisation 6. Experimented custom partitioning successfully to avoid skewed distribution of data across partitions 7. Incorporated novel approaches such as broadcasting smaller datasets to all nodes to preclude data shuffle, arising from sql table joins; Using Kryo serialisation to reduce memory pressure on cluster 8. Tuning & optimisations lead to decrease in query time from 35 minutes to 8 minutes. 9. Developed scripts to automate job executions, scaling nodes in cluster, alerting, reporting System reliably performs 50+ spark sql jobs iteratively on 300+ GB data daily in 3 hours, taking only 40% of original time after optimisations (using 8 r3.xlarge ec2 nodes). Technologies Used - Apache Spark (Core, SQL, Streaming), MongoDB, Maven, Scala, Mesos

1. Travel Itinerary Document Platform: (i) Developed a system where teams can make new itinerary docs or search previous created itineraries docs based on hotels, destinations, activities, and hence recommend relevant itineraries to customer (iI) Implemented features like search, filter, sort, completion suggester based on destinations, hotels 2. Image Search Cloud Tool: (i) Developed a system for searching images from Twitter, Instagram, Google Places etc, based on location, keywords. A subset of images are downloaded & indexed for future retrieval by teams (iI) Teams can now search from a subset of saved quality images, based on location & keywords Technologies Used – Elastic Search, Couchbase, MySql, Spring MVC, Hibernate, Java, Maven

Mathematics & Computing, Big Data Analytics