Data Engineer

People Prime Worldwide

Greater Kolkata Area

Key Responsibilities

Design, develop, and maintain scalable data pipelines using Python and PySpark.

Develop and optimize ETL/ELT workflows for large-scale data processing.

Work with AWS cloud services to build and manage data engineering solutions.

Write complex and optimized SQL queries for data extraction, transformation, and analysis.

Implement data quality, validation, and reconciliation processes.

Troubleshoot and optimize data pipelines for performance and reliability.

Collaborate with Data Scientists, Analysts, Architects, and application teams.

Participate in code reviews, testing, deployment, and production support.

Follow best practices for security, scalability, monitoring, and cost optimization.

Required Skills

Strong programming experience in Python.

Hands-on experience with PySpark / Apache Spark.

Strong SQL development and database concepts.

Good hands-on experience with AWS.

Experience building and supporting ETL/ELT pipelines.

Understanding of data warehousing and data lake concepts.

Strong problem-solving and debugging skills.

Good communication and collaboration skills.