Lead Software Engineer
Key Responsibilities
Design and develop inbound and outbound file processing using AWS Glue, Python, and PySpark.
Develop reusable Python modules, utilities, and frameworks for data ingestion, validation, transformation, and error handling.
Build batch inbound and outbound file processing using Amazon S3 and AWS Glue.
Configure Amazon EventBridge rules for scheduling and event-based processing.
Develop AWS Lambda functions for lightweight processing, automation, validation, and integration activities.
Implement messaging and notification solutions using Amazon SNS and Amazon SQS.
Configure Amazon CloudWatch logs, metrics, dashboards, and alarms for operational monitoring.
Develop workflows using AWS Glue Workflows, AWS Step Functions, or equivalent orchestration services.
Required Skills And Experience
3–8 years of experience in Python development and data engineering.
Strong hands-on experience with AWS Glue and PySpark.
Excellent knowledge of Python programming, object-oriented programming, and exception handling.
Experience designing ETL pipelines and data processing frameworks.
Strong experience with Amazon S3, including bucket structures, lifecycle policies, partitioning, and file formats.
Experience with Amazon EventBridge for scheduled and event-driven execution.
Experience Developing And Deploying AWS Lambda Functions Using Python.
Experience with Amazon SNS and SQS for notifications, messaging, and asynchronous processing.
Experience with Amazon CloudWatch for logging, monitoring, alerting, and troubleshooting.
Understanding of AWS IAM, security policies, encryption, and least-privilege access.
Experience working with relational databases such as Oracle, PostgreSQL, MySQL, or Amazon RDS.
Strong knowledge of SQL and database connectivity from Python or AWS Glue.
Experience working with REST/SOAP APIs and external system integrations.
Understanding of JSON, XML, CSV, Excel, and fixed-width file formats.
Experience with Git and software development best practices.