Data Engineer – AWS Big Data & PySpark

Capgemini Engineering
Chennai, Tamil Nadu, India

Join a team focused on building large-scale data platforms and analytics solutions. As a Data Engineer, you will design, develop, and optimize high-volume data pipelines leveraging AWS cloud services, PySpark, and modern data lake architectures.

Your Role

You will be responsible for developing and maintaining scalable data pipelines, processing large datasets, and enabling reliable data solutions for business analytics and operational reporting. The role involves working with AWS services, ETL/ELT frameworks, orchestration tools, and data lake technologies.

In This Role, You Will:

  • Design, develop, and maintain scalable data pipelines to process data from multiple enterprise platforms.
  • Build, manage, and optimize large-scale ETL/ELT workflows handling high-volume data workloads.
  • Develop data processing applications using PySpark, Spark SQL, and Python on AWS EMR.
  • Create and optimize Athena (Trino/Presto) queries for reporting, troubleshooting, and data analysis.
  • Design, implement, and enhance data lake solutions using Amazon S3, AWS Glue, Athena, and EMR.
  • Collaborate with business analysts, data scientists, and stakeholders to deliver reliable and scalable data solutions.
  • Support migration and modernization initiatives involving Airflow/MWAA and workflow orchestration platforms.
  • Implement data validation, monitoring, and quality checks to ensure data accuracy and consistency.
  • Troubleshoot production pipeline failures, performance bottlenecks, and data quality issues.
  • Follow DevOps and CI/CD best practices using GitLab for code management and deployments.
  • Participate in architecture discussions, code reviews, and technical design activities.
  • Create technical documentation, test plans, and operational runbooks.

Your Profile

Mandatory Skills

  • 6–12 years of experience in Data Engineering and Big Data technologies.
  • Strong hands-on experience with Python, PySpark, and Spark SQL.
  • Expertise in AWS EMR, Amazon S3, AWS Glue, and Athena.
  • Experience building and maintaining large-scale ETL/ELT pipelines.
  • Strong knowledge of data lake architectures and data warehousing concepts.
  • Experience with workflow orchestration tools such as Airflow/MWAA.
  • Strong SQL skills and experience with Trino/Presto/Athena query optimization.
  • Experience with data validation, monitoring, and troubleshooting.
  • Familiarity with GitLab, CI/CD pipelines, and DevOps practices.
  • Strong analytical and problem-solving capabilities.Preferred Skills
  • Experience with telecom, network, billing, provisioning, or large enterprise datasets.
  • Knowledge of real-time and batch data processing frameworks.
  • Exposure to data governance and data quality frameworks.
  • Experience with cloud-native analytics and reporting solutions.
  • Understanding of Agile development methodologies.
  • What You'll Love About Working Here
  • Work on large-scale cloud and big data platforms.
  • Exposure to modern AWS data engineering technologies.
  • Opportunity to solve complex data processing and analytics challenges.
  • Collaborative environment with data scientists, engineers, and business teams.
  • Continuous learning and career growth in cloud and data engineering.

About Capgemini in India

Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organizations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,000 team members in more than 50 countries. We deliver end-to-end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations. The Group reported 2024 global revenues of €22.1 billion.

Capgemini in India comprises nearly 235,000 team members working across 13 locations: Bangalore, Bhubaneswar, Chennai, Coimbatore, Gandhinagar, Gurugram, Hyderabad, Kolkata, Mumbai, Noida, Pune, Salem, and Tiruchirappalli.

Make it real | www.capgemini.com/in-en.

Score my resume against this job, free →

Questions about this job

How do I apply for the Data Engineer – AWS Big Data & PySpark job at Capgemini Engineering?

Use the apply link on this page: you apply directly with Capgemini Engineering on LinkedIn. Before you do, score your resume against this job description on Fitboard. It shows which skills and keywords the job asks for that your resume doesn't show, so you can fix them first.

What skills does Capgemini Engineering want for this Data Engineer – AWS Big Data & PySpark role?

The job description asks for Python, PySpark, Spark SQL, AWS EMR, Amazon S3, AWS Glue, Athena and ETL/ELT. Make sure your resume shows the ones you have, in the words the job uses: applicant tracking systems look for them.

How can I get a job at Capgemini Engineering?

Apply to the Capgemini Engineering roles that match your experience, and tailor your resume to each job description: for this role that means showing Python, PySpark, Spark SQL. A resume that matches the job's own keywords is more likely to get past screening. Fitboard scores your resume against this exact job for free.

Is this Data Engineer – AWS Big Data & PySpark job remote or onsite?

This is based in Chennai, Tamil Nadu.

Is it free to apply? Does Fitboard charge anything?

Applying is free: you apply directly with Capgemini Engineering, and this job page and its apply link cost nothing. Checking your resume against this job is free too, with 3 free scans when you sign in and no credit card needed.

Will you get shortlisted for this role? Check your match free →