🚨Immediate Joiner: Data Engineer - Python | PySpark (6+ yrs)🚨

Innovya Technologies
India

Job Title: Data Engineer - Python | PySpark

Experience: 6+ years

Job Type: India, Full-Time, Remote

Timings: 4pm-1am IST

Innovya Technologies is a fast‑growing software consulting firm specializing in business automation and modern digital solutions. We enable organizations to accelerate value realization from their technology and transformation initiatives through expert, context‑driven product development and high‑quality engineering services.

About the Role:

We are looking for an experienced PySpark Developer with deep expertise data streaming development and deployment to join our dynamic team. A Python developer who will play a key role in maintaining, enhancing, and modernizing critical high volume and real time data pipelines. The ideal candidate will have a strong background in building scalable, fault-tolerant, and high-performance ELT applications, with a keen interest in AI enabled development.

You will work closely with cross-functional teams to design, analyse, build and deploy complex systems, providing technical direction and expertise to ensure the delivery of robust, efficient, and scalable solutions.

Key Responsibilities:

  • Design, develop, and optimize scalable data pipelines using PySpark
  • Process and analyze large datasets in distributed environments
  • Collaborate with data engineers, analysts, and stakeholders to deliver data solutions
  • Write efficient, reusable, and reliable PySpark code
  • Perform data cleansing, transformation, and validation
  • Optimize Spark jobs for performance and cost efficiency
  • Integrate data from various sources (databases, APIs, streaming platforms, etc.)
  • Troubleshoot and debug data pipeline issues
  • Maintain documentation for data workflows and processesRequired Skills:
  • 6+ years of total experience in Python development, with at least 5 years in Apache Spark (PySpark).
  • Hands-on experience with Python and PySpark in production data engineering environments.
  • Use Spark DataFrames for large-scale data transformation and processing.
  • Experience with complex aggregations, window functions, conditional aggregations, and performance tuning in Spark.
  • Good understanding of Spark execution concepts such as partitions, shuffles, join behavior, and optimization techniques.
  • Experience with API integrations, including pagination, request handling, response transformation, and fault tolerance.
  • Working knowledge of AWS, preferably with services such as S3, Lambda, Glue, EMR, Athena, Step Functions, or similar.
  • Experience handling JSON and schema-based parsing in distributed data pipelines.
  • Familiarity with SQL-based data extraction and transformation.
  • Ability to write production-quality, maintainable, and testable code.
  • Good analytical and problem-solving skills, with the ability to understand complex data flows and business rules.
  • Experience identifying bugs, troubleshooting processing issues, performing root cause analysis, and implementing fixes in distributed systems.
  • Ability to create and maintain unit tests to improve code quality and reduce regressions.
  • Preferred - 2+ years of DataBricks and Data Lake development experience

Score my resume against this job, free →

Questions about this job

How do I apply for the 🚨Immediate Joiner: Data Engineer - Python | PySpark (6+ yrs)🚨 job at Innovya Technologies?

Use the apply link on this page: you apply directly with Innovya Technologies on LinkedIn. Before you do, score your resume against this job description on Fitboard. It shows which skills and keywords the job asks for that your resume doesn't show, so you can fix them first.

What skills does Innovya Technologies want for this 🚨Immediate Joiner: Data Engineer - Python | PySpark (6+ yrs)🚨 role?

The job description asks for Python, PySpark, Apache Spark, Spark DataFrames, API integrations, AWS, JSON and SQL. Make sure your resume shows the ones you have, in the words the job uses: applicant tracking systems look for them.

How can I get a job at Innovya Technologies?

Apply to the Innovya Technologies roles that match your experience, and tailor your resume to each job description: for this role that means showing Python, PySpark, Apache Spark. A resume that matches the job's own keywords is more likely to get past screening. Fitboard scores your resume against this exact job for free.

Is this 🚨Immediate Joiner: Data Engineer - Python | PySpark (6+ yrs)🚨 job remote or onsite?

This is based in India.

Is it free to apply? Does Fitboard charge anything?

Applying is free: you apply directly with Innovya Technologies, and this job page and its apply link cost nothing. Checking your resume against this job is free too, with 3 free scans when you sign in and no credit card needed.

Will you get shortlisted for this role? Check your match free →