Srikanth M - Data Engineer
Srikanth M - Data Engineer
Srikanth M - Data Engineer
com
Sr. Data Engineer +1 469 268 8630
SUMMARY OF EXPERIENCE
Senior data engineer with 10 years of experience in building data intensive applications, data analytics,
business intelligence, data integration and migration using SAS, Python, Oracle, Snowflake, ADF, Databricks,
Synapse Analytics, Kafka and DBT.
Expertise in building/migrating data warehouse on Snowflake cloud database.
Experience in creating snowflake warehouses and moving data from traditional databases to Snowflake.
Experience in building data pipelines in Azure Data Factory.
Implemented data skew patterns for removing the data skewness across the partitions.
Implemented Databricks transformations on data using notebooks and provided the configuration in notebook
files for the various stages.
Good exposure in Snowflake Cloud Architecture and SNOWPIPE for continuous data ingestion.
Proficient in understanding business processes/requirements as per user stories and translating into technical
requirements.
Experience in creation of dedicated SQL pools and spark notebooks in synapse analytics.
Good understanding of data storage in synapse SQL pools.
Extensively worked in ETL process consisting of data transformation, data sourcing, mapping, conversion along
with data modelling.
Advanced SQL skills including complex Joins, snowflake stored procedures, clone, views, Materialized views etc.
Experience in building data pipelines in Azure Data Factory.
Experience with Snowflake Datawarehouse, deep understanding of Snowflake architecture and processing.
Created Clone objects to maintain zero copies in Snowflake.
Handling large and complex data sets like JSON, CSV files from various sources like ADF and AWS S3.
Experience in writing complex SQL scripts using Statistical Aggregate functions and Analytical functions to
support ETL in snowflake cloud data warehouse.
Used COPY/INSERT, PUT, GET commands for loading data into Snowflake tables from internal and external
stages.
Good understanding of Kafka topics, consumers, and providers.
Experience in integrating DBT and snowflake.
Created the SQL models in DBT for data movement in snowflake.
Experience in creating Azure Event Hub, Azure Key Vault, Stream Analytics.
Experience of development using software development methodologies such as Agile and Waterfall.
Worked on bulk loading data into Snowflake tables.
Expertise in SAS/Base and SAS/Macros programming.
Excellent understanding of SAS ETL, SAS BI.
Rich hands-on experience in SAS CI Studio, SAS/Data Integration Studio, SAS BI Tools and SAS Enterprise Guide.
Expertise in SAS 9.4 and SAS 9.3 administration activities.
Ability to work independently and as a team with a sense of responsibility, dedication, commitment and with an
urge to learn new technologies.
Excellent Client interaction skills and proven experience in working independently as well as in a team.
Exposure to Power BI concepts and little experience on creation of dashboards.
TECHNICAL EXPERTISE
Cloud Data Warehouse : Snowflake
Cloud ETL& Analytics : Azure Data Factory, Data Bricks, Synapse Analytics
Big Data : HDFS, Hive, Pig, Spark, Airflow
Streaming Tools : Kafka, Azure Stream Analytics
Cloud Environment : Azure, AWS
Programming Language : Python
Database : Oracle, PGSQL, MySQL
Operating Systems : Linux, Windows
Analytical Tools : SAS 9.4 and SAS 9.1.3, SAS Base 9.4, SAS Macros, SAS Management
Console 9.4 and 9.1, SAS Data Integration Studio 3.4 & 7.1, SAS OLAP
Cube Studio 9.1, SAS Information Map Studio 3.1, SAS Web Report
Studio, SAS information Delivery Portal, SAS customer Intelligence
CI/CD tools : Jenkins, GIT, Azure Devops
EDUCATION
Bachelor of Technology in Bioinformatics, Sathyabama University, Chennai, India- 2011
PROFESSIONAL EXPERIENCE
United Health Group (Optum), MN, USA. April 2023 – Till Date
Cloud Data Engineer
Project: Healthcare Economics
Roles and Responsibilities:
Migrated the data from SQL server management to cloud data warehouse snowflake.
Migrated the HIVE quires to Databricks.
Developed and executed the spark jobs to perform data cleaning and business transformation.
Developed notebooks using azure PySpark and established the connection between Databricks and Azure Data
factory.
Implemented transformation logic on delta lake tables and audit log creations.
Developed the ADF pipelines as per the business requirements and written the Databricks notebooks which are
consumed by the ADF pipeline.
Developed Snowflake stored procedures and materialized views as per business needs.
Load the streaming data to Databricks delta lake layer through Kafka.
Good understating of Kafka topics, producers, and consumers.
Integrated Kafka Confluent and snowflake to read streaming data to generate business reports.
Integrated DBT and snowflake to build SQL models in DBT to execute snowflake queries.
Used DBT to debug complex chains of queries. They can split into multiple models that can be tested separately.
Worked on pipeline creating activities using Airflow tool.
Load the data into Azure Synapse Analytics from Azure Data Lake Storage.
Created External tables and Materialized views in Synapse Analytics.
Experience in creation of serverless SQL and dedicated SQL pool.
.
United Health Group (Optum)
Cloud Data Engineer Feb 2023 – Apr 2023
Project: SMART Modernization
Roles and Responsibilities:
Converted legacy oracle procedures based ETL processes to snowflake SQL procedures.
Designed ADF ETL pipelines to orchestrate the snowflake procedures.
Implemented the email mechanism in ADF by using Microsoft Graph API.
Integrated Kafka and snowflake to consume the streaming data.
Created Airflow DAGS to schedule the ingestion, ETL jobs and various business reports.
Redesign and optimize the performance of existing snowflake procedures.
Extract transforms and loads data from source systems to Azure data storage using an Azure Data Factory, Spark
SQL and process the data in data bricks.
Analyze and solve business problems at their root, stepping back to understand the broader context.
Developed a data validation framework, resulting in an improvement of data quality.
Worked on Snowflake streams to process incremental records.
Addressing data issues and providing permanent fixes.
Created materialized views to speed up the query processing for the rarely updated large tables.
Used Temporary and Transient Data Objects on different datasets.
Used the copy command to load bulk data into snowflake from various sources.
Create and manage the Snow pipe for continuous data loading.
Zero Copy cloning – Cloning databases for Dev and QA environments.
Estimating requirements and committing deadlines with business.
Converting design documents into technical specification.
Environment: Snowflake, SQL, Azure Data Factory, Azure Data Bricks, JIRA, Postman, SAP BW, Azure Blob.
Environment: Snowflake, AWS S3, Oracle, SQL, MySQL, PGSQL, SAS DI, SAS SMC, SAS LSF, SAS FM, SAS OLAP
Environment: Oracle, SQL, MySQL, PGSQL, SAS DI, SAS EG, SAS SMC, SAS LSF, SAS FM, SAS OLAP, Linux