Vikram Adithya

Senior Databricks Engineer

Buffalo, US.

About

Senior Databricks Engineer with over 10 years of expertise in designing, developing, and optimizing scalable data engineering and cloud solutions across diverse industries including healthcare, retail, and financial services. Proven ability to architect high-volume ETL/ELT pipelines, Lakehouse environments, and advanced customer data platforms (AEP, CDP, AJO) using Databricks, Spark, Python, SQL, AWS, and Azure. Adept at leveraging data modeling, performance optimization, and CI/CD practices to deliver robust, high-quality data solutions that drive real-time customer activation and business intelligence.

Work

BCBS

|

Sr Databricks Engineer

Florida, FL, US

→

Summary

Led the design and implementation of scalable data solutions for enterprise customer data platforms and personalized customer experiences within the Adobe Experience Platform ecosystem.

Highlights

Architected and optimized high-performance data engineering solutions for Adobe Experience Platform (AEP), Adobe CDP, and Real-Time CDP, enabling real-time customer activation and comprehensive Customer 360 views.

Led the development of enterprise-scale ETL/ELT pipelines using Databricks, Spark, Python, SQL, Delta Lake, ADLS, and AWS S3, processing large volumes of structured and semi-structured customer data.

Implemented robust Delta Lake architectures, leveraging ACID transactions, schema evolution, and incremental processing patterns to ensure data reliability and optimize storage for critical customer data.

Engineered scalable Spark/PySpark applications and Databricks notebooks, optimizing distributed data processing and complex transformations across large enterprise datasets for improved efficiency.

Collaborated with Adobe Architects, AJO Engineers, Marketing Technology teams, and business stakeholders to translate complex business requirements into scalable data architectures and solutions.

MasterCard

|

Databricks Engineer

New York, NY, US

→

Summary

Developed and maintained scalable data pipelines and solutions for enterprise customer data initiatives, processing large volumes of customer, transaction, and behavioral data.

Highlights

Engineered and optimized scalable data pipelines using Databricks, Spark, PySpark, Python, SQL, and Delta Lake, processing large volumes of customer, transaction, and behavioral data for enterprise initiatives.

Designed and maintained end-to-end ETL/ELT pipelines, ingesting data from diverse enterprise sources to create reliable datasets for analytics, customer profiles, and marketing applications.

Developed robust data processing workflows in Databricks using PySpark and SQL, implementing advanced data cleansing, transformation, and incremental processing for large-scale datasets.

Integrated and prepared customer data for unified profiles, identity resolution, segmentation, and real-time activation within Adobe Experience Platform (AEP) and Adobe CDP.

Wells Fargo

|

Data Engineer

Charlotte, NC, US

→

Summary

Designed and implemented scalable data models and ETL pipelines for enterprise reporting and analytics, leveraging Databricks Lakehouse architecture.

Highlights

Designed and implemented scalable conceptual, logical, and physical data models for enterprise reporting and analytics, utilizing Databricks Lakehouse architecture to enhance data accessibility.

Developed robust Bronze, Silver, and Gold data layers with Databricks and Delta Lake, ensuring reliable and analytics-ready data products for downstream consumption.

Translated complex business and reporting requirements into scalable data structures and dimensional models, enhancing reporting and analytics capabilities.

Built and optimized ETL/ELT pipelines using AWS Glue, PySpark, Python, and SQL, efficiently processing large volumes of structured and semi-structured data for reporting.

LTI Mindtree

|

Data Engineer

Mumbai, Maharashtra, India

→

Summary

Designed and developed scalable data models and ETL pipelines to support enterprise reporting, analytics, and business intelligence requirements.

Highlights

Designed and developed scalable data models and ETL pipelines, supporting enterprise reporting, analytics, and business intelligence requirements for enhanced decision-making.

Constructed robust data pipelines using Apache Spark, PySpark, Hive, Python, and SQL, processing large volumes of structured and semi-structured data.

Managed and maintained Bronze, Silver, and Gold data processing layers, ensuring organized and reporting-ready datasets for various business needs.

Optimized large-scale data processing performance by applying data partitioning, bucketing, file-format optimization, and Spark tuning techniques.

ESM Square Technologies

|

Data Engineer

Hyderabad, Telangana, India

→

Summary

Developed and maintained enterprise ETL pipelines and data integration workflows to support reporting, analytics, and business intelligence requirements.

Highlights

Developed and maintained enterprise ETL pipelines and data integration workflows, enhancing reporting, analytics, and business intelligence capabilities.

Designed and implemented logical and physical data structures for analytical databases, aligning with business requirements to improve data accessibility.

Engineered ETL workflows using Informatica PowerCenter, SQL, Python, and Shell scripting to efficiently extract, transform, and load data from diverse source systems.

Improved overall data quality by implementing robust data cleansing, standardization, transformation, and business-rule validation processes.

Skills

Data Engineering & Lakehouse

Databricks, Apache Spark, PySpark, Delta Lake, ETL/ELT, Lakehouse Architecture, Data Processing, Batch Processing, Incremental Processing.

Programming & Query Languages

Python, SQL, PL/SQL, Shell Scripting.

Cloud Platforms

Microsoft Azure, AWS, Azure Data Lake Storage (ADLS), AWS S3, AWS Redshift, AWS Glue, AWS EMR, AWS Athena.

Data Integration & Orchestration

Azure Data Factory (ADF), Apache Airflow, Oozie, Informatica PowerCenter, REST APIs, API Integration, Kafka, Event Streaming.

Adobe Experience Platform

Adobe Experience Platform (AEP), Adobe Customer Data Platform (CDP), Adobe Real-Time CDP, Adobe Journey Optimizer (AJO), Customer 360, Customer Identity Resolution, Customer Profile, Audience Segmentation, Real-Time Activation.

Data Modeling & Warehousing

Data Modeling, Dimensional Modeling, Conceptual/Logical/Physical Data Modeling, Star Schema, Fact & Dimension Tables, Slowly Changing Dimensions (SCD), Data Marts, Data Warehousing, Source-to-Target Mapping, Data Lineage.

Databricks & Data Governance

Unity Catalog, Delta Lake, ACID Transactions, Schema Evolution, Schema Enforcement, Data Governance, Access Control, Data Quality, Data Validation, Data Reconciliation.

Data Engineering & Optimization

Spark Optimization, Partitioning, Bucketing, Caching, Efficient Joins, Query Optimization, File-Size Optimization, Performance Tuning, Fault Handling, Error Handling.

Analytics & BI

Power BI, Tableau, Analytical Datasets, Reporting, Business Intelligence.

DevOps & CI/CD

Git, Azure DevOps, Jenkins, CI/CD, Branching, Pull Requests, Code Reviews, Build & Release Management, Automated Deployment.

Big Data Technologies

Hadoop, HDFS, Hive, Sqoop, Apache Spark, Kafka.

Databases

Oracle, SQL Server, PostgreSQL, Snowflake, AWS Redshift.

Machine Learning

ML Feature Engineering, Customer/Behavioral Feature Engineering, Machine-Learning-Ready Data Pipelines.

Containers & Development

Docker, Kubernetes, Linux.

Project & Delivery

JIRA, Agile, Scrum, Production Support, Root Cause Analysis, Technical Documentation.