Harshit Singhai

Data Engineer | Developer | Software Engineer | ML

A little about me,

About

Hello!

My name is Harshit. I’m a software engineer from 🇮🇳 I'm an open-source enthusiast, & a learner. I'm deeply passionate about software development & innovations. I like to talk about software development, MLOps, data engineering, scalable appliation, infrastructure, devops, and cloud.

My Skills

Data Engineering

Backend

Software Engineer

Machine Learning

Education

Academics

Bachelor of Technology in Computer Science & Engineering

Bennett University

2016 - 2020

Main coursework: Data Structures, Design and analysis of Algorithms, Machine Learning Deep Learning Applied Machine Learning, Cloud Computing, Database Systems, Operating Systems, Software Engineering.

Experience

Profile

Data Engineer

Deloitte USI

February 2022 - Ongoing

  • Collaborated with global asset leader’s internal audit team, architecting cloud analytics for risk detection and compliance
  • Eliminated inter-job dependency failures and mismanaged cron jobs by orchestrating cross-account data pipelines with Cloud
  • Architected secure PII infrastructure with masking, enhancing compliance, saving $50k/year in potential fines
  • Saved 52 man-hours per week by migrating auditor reporting to AWS, and eliminating the need for manual reporting scripts.
  • Redshift Spectrum, enabling direct S3 queries, minimized ETL, and efficient transformations of historical data
  • Enhanced data availability through robust pipeline design, achieving 99.8% uptime, efficient ingestion from 7+ data sources
  • Built ML-powered fraud detection using SageMaker model, saving $2M from averted fraud
  • Drove 40% S3 cost reduction by Intelligent Tiering, shifting pipelines to serverless AWS Glue with autoscaling, replacing EMR
  • Maximised analytics efficiency by replacing manual data transfers with AWS DMS synchronization, reducing overhead
  • Built trust with Glue data quality: automated checks on trends, alerts for anomalies, ensuring reliable data for data governance

    • Software Engineer - Platform

    Quantive

    Auguest 2021 - January 2022

  • Built microservices pipelines, enabling real-time anomaly detection and OKR performance analysis
  • Optimized microservices leading to 30% reduction in memory usage & processing time through memory profiling
  • Augmented microservice built on Pandas, using efficient vectorized operations to deliver KPI insights 20% faster
  • Increased data throughput by 40% using Celery-based async processing, RabbitMQ as message broker, and ClickHouse

    • Software Engineer - ML

    Deepsource

    Dec 2020 - June 2021

  • Built AI recommendation system to guide users towards the most impactful code issues, enhancing code quality
  • Created 50k+ ML models, to predict high-impact code issues based on user’s historical data, saving developers’ 30 hours/week
  • Leveraged Kubeflow orchestration and Katib for hyperparameter tuning, achieving 20% faster ML pipeline training.
  • Implemented adaptive ML model retraining, adapting to new patterns, and ensuring model relevancy
  • Prioritized code quality by deploying only model versions demonstrating superior precision and ROC AUC

    • MLH Fellow

    Major League Hacking

    Octo 2020 - Dec 2020

  • Part of the inaugural class of MLH Fellows (powered by GitHub & Facebook), 130 students selected out of 30,000.
  • Contributed to Open Source projects written in Python with a focus on Python & the Python ecosystem with a team of Fellows under the educational mentorship of a professional software engineer.
  • Contributed to BentoML - From trained ML models to production-grade prediction services with just a few lines of code (https://www.bentoml.ai/)
  • Currently contributing to BentoML

    • Software Engineer Intern

    Finception/Finshots

    Jan 2020 - Jun 2020

    Finception is a Bengaluru based FinTech startup Backed by Zerodha who aims to build Robo-advisor financial planner for millennials.

    The key responsibilities that I took charge of were:
  • Developed the user-interface for both web app and cross-platform mobile app (Android and iOS) using React and React Native.
  • Worked with Redux, Sass, Material UI, and Typescript to write reusable and maintainable code.
  • Developed several API endpoints using Django, Django REST framework, and PostgreSQL database.
  • Utilized Python libraries like Pandas, Numpy, Matplotlib for processing tabular format data.
  • Worked as data science, analyzing and driving insights from the Finshots newsletter Mailchimp data
  • Used Pandas, Matplotlib, Seaborn and other libraries for data analysis

    • Technologies used: React, React Native, Javascript, Typescript, Python, Django, PostgreSQL, Git, Pandas

    Software Engineer Intern

    Leena AI

    May 2019 - July 2019

    Leena AI, a member of the Y Combinator Summer 2018 class, wants to change that by building HR bots to answer questions for employees instantly.

    The key responsibilities that I took charge of were:
  • Creating a standalone web application ffor the HR dashboard allowing HR to schedule, view, and delete messages.
  • Integrated the standalone application with Leena AI chatbot for better usability resulting in enhanced HR experience.
  • Implemented Socket.io in node.js and react for real-time message status and delivery.
  • Reported directly to the frontend lead to create and maintain internal react libraries used by the development team.
  • Created a CRUD application using React, Node, and MongoDB for the HR web interface and implemented different functionalities for the HR admin.

    • Technologies used: React, Javascript, Typescript, Node.js, Express, MongoDB, Git

    Software Engineer Intern

    Opalina Technologies

    June 2018 - August 2018

    Worked as a Software Developer Intern at Opalina Technologies. The key responsibilities that I took charge of were:
  • Designing the recruitment platform, a web application for conducting tests, keep track of scoring, and assessing the results.
  • Worked as a Front-end Developer using React, Typescript, Antd, and Material UI
  • Used Redux, Cache, and Local Storage.
  • Wrote react internal library used by other developers working on the project
  • Created reusable components
  • Creating REST API handling methods/features for user role 'Test Taker': Test Management (test-taking environment).

    • Technologies used: React, Typescript, Antd, Material UI

    Side Projects

    Projects

    WeBase - Making Machine Learning Simple

    WeBase

    React, Node.js, Express, Mongodb, Flask, Keras, Tensorflow, Machine Learning, NLP, Computer Vision, Sklearn

    Aug 2019 - Dec 2019

  • WeBase provides common machine learning models as a service to make advance AI accessible for developers. Our simple to use user-interface allows users to use our pre-trained machine learning models with ease.
  • Used Nodejs and Express framework to build server side logic, applied middlewares to handle http requests, used flask to deploy and serve ml models.
  • Applied JWT middleware for Authorization. Flask and Keras for deploying, creating and serving ML models.
  • Interpreted 20+ image datasets for image classification algorithms. Extensively worked with transfer learning using pre-trained model.
  • Web App Demo (Test User credentials - harshitsinghai77@gmail.com, password: Pass@12345)
  • Read more at Fictionally Irrelevant

    • Nexmo - Multilingual AI voice Interface for Kotak Mahindra Bank (SIH, 2019)

    National Finalist, Smart India Hackathon 2019

    Vue, Node.js, Progressive Web App, Dialogflow

    March 2019

  • Successfully improvised Kotak Mahindra Bank’s virtual assistant, Keya by giving it multi-lingual voice capability; providing it a better natural language processing and context-based answering capabilities as part of Smart India Hackathon 2019

    • Flower Recognition Data Science Competition Hackearth

    HackerEarth Competition

    Python, Keras, Tensorflow, Computer Vision, Image classification, Transfer learning, Pre-trained model

    Oct 2019

  • Implemented a multi-label image classification model using Keras. Applied concepts like image augmentation and used pre-trained models

    • Peach Pixel - Turn Design Mockups into HTML Code using Deep Learning

    React, Flask, Keras, Tensorflow, NLP, Computer Vision, Deep Learning

    Oct 2019

  • Built web application for front-end automation using deep learning. Engineered a neural network to code a basic HTML, CSS and Boot- strap website based on a picture of a design mockup leading to rapid design prototyping, thus increasing productivity and saving time. https://bit.ly/3hJPp8x

    • Coronavirus News, Updates, and Analytic Dashboard

    React, Redux, Typescript, Node.js, REST API

    Oct 2019

    Microsoft Student Partner

    Microsoft

    September 2019 - December 2019

  • Part of a global group of campus leaders.
  • Hands-on workshops across India onDevOps, Open Source, Cloud, ML, Web, etc

    • Accomplishments & Awards

    Accomplishments

    National Finalist

    Smart India Hackathon 2019 - Kotak Mahindra Bank

    March 2019

  • Multilingual AI voice Interface for Kotak Mahindra Bank
  • Improvised Kotak Mahindra Bank’s virtual assistant, Keya by giving it multi-lingual voice capability; providing it a better natural language processing and context-based answering capabilities

    • Secured 1st Position

  • MLH 2018 Local Hackday Hackathon hosted by GitHub

    • Secured 3rd position

  • HackBMU hackathon, BML Munjal University, Gurugram

    • Project mentor - Bennett University

  • Freshman Project Mentor for Bennett University freshers for the year 2019
  • Conducted and managed 30+ labs sessions and tutorials

    • Some of my blogs

    Blogs

    July 16, 2023

    DuckDB - Accelerating Data Analysis with Speed and Simplicity

    July 31, 2023

    Sky Computing: Unleashing the Cloud's Full Potential

    November 27, 2022

    Data Engineer in 2022

    December 15, 2023

    Side Projects 2023

    November 1, 2023

    The Data Analytics Lifecycle

    July 22, 2021

    Be productive with Nemo

    Let's Talk

    Contact

    Want to connect?
    My inbox is always open!

    May the Source be with you 🚀