Data Engineer (Azure, Snowflake, dbt)

Data Engineer (Azure, Snowflake, dbt) | 6-10 Years

Description:

Data Engineer is responsible for the hands-on development, maintenance, and optimization of robust, end-to-end (E2E) analytics data pipelines. You will leverage a modern cloud stack (Azure, Snowflake, dbt) to ingest and transform data according to Data Vault 2.0 standards, while actively utilizing AI-assisted development tools to write high-quality code faster and more efficiently.

Responsibilities

• Build and maintain scalable, automated data ingestion and processing pipelines using Azure, dbt, Snowflake to extract data from various source systems and load it into Snowflake for further processing.
• Data Transformation & Modeling: Use dbt (data build tool) to develop modular, testable SQL transformations. Implement physical Data Vault 2.0 structures (Hubs, Links, Satellites) based on architectural designs.
• AI-Assisted Engineering: Actively use AI coding assistants (e.g., GitHub Copilot / Cursor) to generate boilerplate code, write dbt tests, draft documentation, and troubleshoot SQL/Python scripts, significantly accelerating your delivery cycle.
• Quality & Testing: Ensure high data quality by writing comprehensive dbt tests and implementing automated data validation checks within the E2E architecture.
• Collaboration: Work closely with the Tech Lead, Data Architect, and business analysts in an Agile environment to understand requirements and deliver reliable data sets for downstream reporting.

Typical Activities:

Writing and optimizing SQL queries and Python scripts for data extraction and transformation.

• Configuring Azure Data Factory, Snowpark Python pipelines for secure, scheduled data movement.
• Developing dbt models, macros, and tests to populate the Snowflake Data Vault.
• Collaborating in daily Scrum meetings and participating in code reviews with the Tech Lead.

Requirements

• Experience: 6-9 years of hands-on experience in Data Engineering or Database Development.
• Technical Expertise: Strong proficiency in writing complex, optimized SQL and solid scripting skills in Python.
• Hands-on experience building pipelines in Microsoft Azure (ADF, Data Lake Storage).
• Practical experience developing and performance-tuning within Snowflake.
• Experience using dbt for data transformation, testing, and documentation.
• Familiarity with incorporating AI-assisted development tools (like GitHub Copilot) into daily coding workflows.
• Understanding of Data Vault 2.0 concepts (Hubs, Links, Satellites) and how to populate them.

Nice to Haves:

Microsoft Certified: Azure Data Engineer Associate or SnowPro Core Certification.

• Experience with CI/CD pipelines (e.g., Azure DevOps, GitHub Actions) and version control (Git).
• Exposure to building dimensional models (Star Schemas) on top of a Data Vault.

Email - abhishek.goswami@thedigitalloom.com