Location: Pune

Experience: 5–7 Years

About the Role

We are looking for anexperienced Senior Data Engineer with 5–7 years of hands-on expertise inbuilding, optimizing, and managing cloud-native data pipelines andarchitectures. The ideal candidate will bring strong proficiency in Azure DataServices, Databricks, PySpark, Python, and SQL, along with the ability toarchitect scalable data solutions, lead best practices, and mentor juniorengineers. You will play a critical role in driving large-scale dataintegration, advanced analytics, and performance optimization across a moderncloud data platform.  

‍

Key Responsibilities  

• Design,architect, and implement scalable, efficient data pipelines using Databricksand PySpark.

• Workextensively with Azure Data Services such as Data Factory, Data Lake, SynapseAnalytics, Functions, and Logic Apps.

• Leadthe automation and optimization of data workflows in Python for transformation,cleansing, and enrichment.

• Write,optimize, and performance-tune complex SQL queries for structured,semi-structured, and unstructured data.

• Designand maintain data models, warehouse schemas, and optimized data structures foranalytical and operational workloads.

• Driveadoption of data governance, security, and compliance standards across datasolutions.

• Collaborateclosely with business stakeholders, solution architects, and data scientists todeliver end-to-end solutions.

• Mentorjunior engineers and contribute to technical leadership and code reviews.

• Participateand contribute to Agile ceremonies including sprint planning, retrospectives,and backlog refinement.

‍

Mandatory Skills

• Strongexpertise in Azure Data Services (ADF, ADLS, Synapse, Azure Functions, LogicApps)

• Advancedproficiency with Databricks (Spark-based development)

• Deephands-on experience in PySpark for distributed data processing and optimization

• StrongPython programming skills for automation, data transformations, and APIs

• AdvancedSQL (query optimization, indexing, partitioning, CTEs, window functions,performance tuning)

• Solidunderstanding of Data Warehousing, Data Modelling, and ETL/ELT best practices

• Experiencedesigning scalable, secure, and high-performance data pipelines

• Hands-onexposure to CDC (Change Data Capture) and near real-time ingestion patterns(e.g., Snowpipe, Kafka)

• Familiaritywith DevOps practices (Git, GitHub, Azure DevOps, CI/CD pipelines, Docker,Kubernetes)

• Experienceworking in Agile delivery and leading sprint-level initiatives

‍

Desired Skills

• Experiencewith Snowflake or other modern cloud data warehouses

• Knowledgeof data governance frameworks (e.g., dynamic data masking, secure views, RBAC)

• Proficiencywith workflow orchestration tools like Apache Airflow, Azure Data Factorytriggers, or Control-M

• Knowledgeof streaming technologies (Kafka, Event Hubs, Spark Structured Streaming)

• Familiaritywith machine learning data pipelines or feature engineering in Databricks

• Exposureto AWS cloud services or multi-cloud environments

‍

Preferred Qualifications  

• Provenexperience in leading and mentoring junior engineers

• Strongproblem-solving and debugging skills for large-scale distributed systems

• Abilityto translate business needs into scalable and efficient data solutions

• Excellentcommunication and stakeholder management skills

• Passionfor data engineering, cloud innovation, and emerging technologies

• Trackrecord of delivering high-performance, cost-efficient, and secure datasolutions

‍