Data Engineer

Job Title: Big Data Engineer.

Location: Hyderabad (Hybrid)

Experience: 8+ Years

Job Type: FTE

Key Skills: Big Data, Delta Lake, Hive, Power BI, Python, SQL, Kafka, Spark.

Key Responsibilities:

• Work closely with technology partners on feature definition, business case development, and overall direction of the enterprise data services platform.
• Ensure platform is operating efficiently and effectively
• Oversee and lead software development efforts
• Provide guidance on routine to highly complex problems to team members
• Align to enterprise software development practices

Responsibilities:

• Hands on development experience in writing Apache/Pi Spark jobs using Python and Scala
• Complete knowledge of the Hadoop ecosystem
• Design and development of applications for Ratings Advance analytical, data innovation systems
• Participate in multiple software development processes including Coding, Testing, De-bugging & Documentation.
• Develop software applications based on clear business specifications.
• Work on new initiatives and support existing Ratings applications.
• Perform Application & System Performance tuning and troubleshoot performance issues.
• Integrate in-house applications with various vendor software platforms.
• Setup development environment / sandbox for application development.
• Check-in application code changes into the source repository.
• Perform unit testing of application code and fix errors.
• Effectively interact with customers, business users and IT staff

Qualifications:

• Master’s degree in Computer Science, Information Systems or Engineering is required, or in lieu, a demonstrated equivalence in work experience.
• 8 + years of IT experience in application development and support.
• 5+ years of Strong experience with Apache/Pi Spark jobs using Python and Scala
• Strong experience with Oracle and related technologies
• Strong experience with SQL and no SQL database technologies.
• Experience with Unix/Linux Operating System.
• Experience using system tools, source control systems, utilities and third-party products.
• Experience using scheduling technologies (i.e. Autosys)
• Experience with DevOps Tools i.e. Git, Jenkins, Ansible etc…
• Experience working in an Agile development environment is MUST
• Experience with Cloud Solutions, Design patterns, data models & data architecture & design is a plus.
• Experience with Change Management and defect tracking tools
• Experience in leading and managing small teams.
• Experience with Java Technologies is a plus
• Financial experience is preferred