I am Mr Nano (my nick name, thanks friends:) ), thanks for visiting my space! I've also been writing at obsidian (feel free to check it out, best viewed on large screens) Anyhow, this is me at a glance:
- Developer Trainer - I enjoy training other developers on upcoming technologies, timeless practices and consultation mindset.
- Big Data Pipelines - I have spent a fair bit of time building large ETL pipelines with ~ 20 workflows and 300+ steps.
- CI/CD Pipelines - Well, I have been working with ThoughtWorks, so that's bread and butter.
- An Aspiring Data Scientist - I'm no data scientist today, but someone who can work with them to bring ideas into production. I had my fun with Genetic algorithms and Linear Optimization.
- Functional Programming Enthusiast - While teaching OOP practises makes me money, believe me it's not a betrayal!
- Managing Infrastructure - While this doesn't excite me as much, if time comes, I have managed a couple of hadoop clusters with ~20 machines.
- Life Long Learner - Could not help not putting this, I'm an avid learner whose constantly exploring this world and trying to learn more than yesterday.
- One of my lifelong passions is training fellow developers on upcoming technologies, timeless practices and consultation mindset. I guess that's my way of learning too.
- I specialize in 2 kind of trainings
- Xtreme Programming and Clean Code
- For anyone who wants to be a "better" developer
- Building and maintaining Large Data Pipelines
- For folks who have some industry experience and want to get acquainted with the in's and out's of Data Engineering
- I have worked with 2 clients for building their data pipelines.
- One of them is a large equipment manufacturer.
- Goldman Sachs is the other proud one.
- Technologies, frameworks, languages and stuff I worked with:
- Hadoop, HDFS, Cloudera, Cloudera Manager, Hive, Oozie, Sqoop, Impala, MapReduce, Avro, Parquet, Spark, Structured Spark Streaming, AWS, AWS S3, AWS EMR, Terraform, Docker, Vault, Airflow, Java, Scala, Gradle, Maven, SBT, Kafka, Postgres
- Scale I worked with
- Build and maintain pipeline for handling 300TB+ data on weekly basis
- Productionize a predictive model to respond in web scale times
- Well, been working with ThoughtWorks, so that's bread and butter.
- In the beginning of time (at ThoughtWorks) gaush! I really struggled with the whole idea of CI / CD pipelines.
- It's satisfying to look back and relish that I (of course with an awesome team) have been able to implement long-lasting pipelines with Jenkins and more recently with CircleCI and Gitlab CI.
- I'm no data scientist today, but someone who can work with them to bring ideas into production. I had my fun with Genetic algorithms and Linear Optimization.
- I have worked with a close friend and an awesome data scientist on my team. We experimented about 6 different models to solve an optimization problem, finally settling on a customized Genetic algorithm.
- We innovated and modeled an enterprise's inventory as supply/demand constraints and optimized them to maximize the yield. What fun!
- These days, I'm learning about Chatbots and NLP. Stay tuned :)
- While teaching OOP practises makes me money, believe me it's not a betrayal!
- As long as I remember, I have been attracted to ideas from functional programming.
- My first exposure to it came from the book SICP and subsequently a course on Coursera about programming languages.
- I use Scala to express myself. I have spent decent time nerding over its type system, functional constructs, pure functions and these days over monadic structures.
- Will love to have a beer/wine/coffee about any of this.
- Honestly, I am interested by everything technical under the Sun (from type systems to assembly to big data to ...), though not so much when it comes to operate infrastructure myself.
- Though as luck will have it, its one of the first thing I did in my enterprise journey.
- This invovled managing 2 clusters, ~20 machines each. We used the Cloudera Manager mostly.
- Later, as the world made everyone an infrastructure engineer, I got exposed to Terraform and today can navigate through it.
- There are a few things one cares very deeply about. Learning is one of those things for me.
- I constantly obsess with ways of learnings better. This shows in my hunt of improving my sessions, my relationship with my trainees or mentees, or simply in the way I journal my thoughts.
- So far I have experience in managing ideas with practising and promoting spaced repetition using Anki.
- More recently I have been experiementing with Graph note taking through Obsidian. It's a life changer!