D
durgaprasadpr718's photo
Durga Prasad Choudhury
From India 05:23 AM (GMT+05:30)
$22/hr or $50,000/yr

Active over a week ago


Share this profile:

Senior Data Engineer/ Analytical Data Engineer

Data Engineer
Years of experience
5+ years
Experience level
Mid-level
Available for
Full-time, Part-time, Contract, Freelance
Available from
12 Apr 2026
Download Resume / CV

Results-oriented Data Engineer with over 4.6+ years of experience delivering scalable, cloud-native data solutions across AWS, Microsoft Azure, and Google Cloud Platform (GCP). Certified in all three platforms, I specialize in PySpark, DBT, Snowflake, Python, and SQL to design, build, and optimize data pipelines that drive real-time analytics and data-driven decisions.

I excel in end-to-end pipeline development, from data ingestion and transformation to validation and optimization, ensuring data quality, performance, and scalability. With strong data modeling expertise and a focus on business impact, I am committed to building efficient, high-performance data architectures.

With a proven track record of managing complex workflows and enabling actionable insights, I help organizations leverage data for smarter, faster decision-making.

Languages

Employment History

Analytics Engineer at Fabletics, LLC 2025 - 2026
Lead analytics engineering for Blue Yonder post-go-live data integrations. Designed and maintained scalable data models in Snowflake to support Merch Planning and supply -chain analytics. Owned production ETL/ELT pipelines, resolving complex data quality and performance issues. Collaborated with business stakeholders, internal platform teams, and external Blue Yonder engineers to deliver stable, long-term solutions. Built monitoring and automation to improve reliability and reduce operational incidents. Supported expansion into Demand and Fulfillment modules and mentored junior engineers.
Analytical Data Engineer II at Softborne Technology Solutions Pte. Ltd. 2023 - 2025
Collaborated with the product management team to identify and prioritize new feature opportunities, leading to timely implementation. Designed and optimized scalable data models in Reactor Data using Snowflake and BigQuery, improving query performance and reducing processing time. Developed and maintained real-time and batch data pipelines using BigQuery, Dataflow, Pub/Sub, and Cloud Composer to support data availability in dashboards and analytics. Built Python-based transformations and leveraged Apache Beam and Spark for large-scale data processing. Utilized Domain Specific Language (DSL) for accurately mapping source data to destination schemas. Used Source Links to join and map multiple data sources into unified schemas. Tuned complex SQL queries in Snowflake and Big Query to optimize performance and manage large-scale retail datasets. Worked closely with cross-functional teams to implement tailored data solutions. Conducted thorough validation and reconciliation checks between source systems and data warehouse to ensure 99%+ data accuracy. Built custom RESTful API connectors using Airbyte for ingesting external data sources. Deployed and managed connectors using Cloud Functions and Cloud Run for scalable and serverless execution. Developed dynamic dashboards to track KPIs such as sales performance and customer lifetime value. Created and maintained transformation logic using DBT, promoting modular, testable, and reusable data models.
Data Engineer I at KPI Partners 2022 - 2023
Created data pipeline using Apache Airflow, Python, and AWS to store weekly raw data. Designed a multi-user, multi-database app for processing thousands of records. Managed SQL and NoSQL data modelling using Big Query, Snowflake, and Amazon DynamoDB for scalable data collection and structured storage. Collaborated with cross-functional teams for pipeline design and business needs. Developed scalable data ingestion processes and optimized ETL workflows. Implemented data quality checks and monitoring mechanisms for data integrity.
Associate Professional Software Engineer at DXC Technology 2021 - 2022
Developed and maintained scalable data pipelines using Azure Data Factory and Databricks. Optimized data storage and retrieval using Azure Synapse Analytics and Delta Lake. Implemented data security best practices with Azure Key Vault and Role-Based Access Control (RBAC). Monitored and troubleshot data pipeline failures using Azure Monitor and Log Analytics. Designed and implemented data models for efficient querying and reporting. Automated ETL processes to streamline data ingestion and transformation. Collaborated with data analysts and stakeholders to ensure data accuracy and consistency. Conducted performance tuning of SQL queries and Spark jobs for optimal execution.

Education

Bachelor of Technology (B.Tech) in Electronics and Communication Engineering (ECE) at Jawaharlal Nehru Technological University Kakinada 2017 - 2021