Back End Developer

Senior Backend Engineer – APIs, Microservices & ML Inference Infrastructure

Position SummaryWe are seeking an experienced Senior Backend Engineer with 7–10 years of software development experience to design, build, and maintain scalable backend services, APIs, and distributed systems.

The ideal candidate will have strong experience developing backend APIs and microservices, working with databases, storage systems, queues, and asynchronous processing frameworks. The role will also support backend orchestration for machine learning inference workloads, including request scheduling, job management, resource utilization, and integration with model-serving infrastructure.

The engineer will work closely with application, platform, machine learning, DevOps, and product teams to build reliable, scalable, and maintainable backend systems.

Success in This RoleA successful candidate will be a strong backend and distributed-systems engineer who can design reliable APIs and services, build scalable asynchronous processing systems, and effectively work across databases, storage, queues, and infrastructure.

Experience supporting machine learning inference services and compute-intensive workloads will be valuable, but the primary focus of the role is designing and delivering robust backend platforms and services.

Key Responsibilities· Design, develop, test, deploy, and maintain scalable backend APIs and microservices.

• Build robust RESTful APIs and service interfaces for mobile, web, and internal applications.
• Design modular, reusable, and maintainable backend service architectures.
• Design and maintain multi-tenant backend systems, enforcing tenant isolation across data, compute, and storage layers.
• Develop asynchronous and event-driven workflows using message queues, task queues, and background workers.
• Design and integrate backend services with:

o Relational and NoSQL databases

o Object and file storage systems

o Caching layers

o Message queues and event-processing systems

o External and internal APIs

• Design efficient database schemas, queries, indexing strategies, and data-access patterns.
• Develop systems for handling high-throughput and concurrent workloads.
• Implement appropriate caching, retry, timeout, rate-limiting, and fault-tolerance mechanisms.
• Support orchestration and scheduling of machine learning inference workloads.
• Develop backend services that manage inference requests, job queues, worker allocation, batching, prioritization, and result handling.
• Integrate backend services with model-serving platforms and inference endpoints.
• Work with ML and infrastructure devs to improve inference throughput, latency, scalability, and resource utilization.
• Develop reliable mechanisms for managing long-running and asynchronous processing jobs.
• Implement application logging, monitoring, metrics, tracing, and error handling.
• Troubleshoot distributed system issues across application, database, queue, storage, and infrastructure layers.
• Write and maintain unit, integration, API, and system-level tests.
• Perform API and service validation using tools such as cURL, Postman, Bruno, or similar API testing tools.
• Develop automated test workflows for APIs and backend services.
• Participate in architecture reviews, code reviews, technical design discussions, and engineering best-practice development.
• Collaborate with DevOps and infrastructure teams on deployment, scaling, observability, and production support.

Required Qualifications· Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Software Engineering, or a related technical discipline.

• 7–10 years of professional software development experience, with significant experience developing backend systems and distributed services.
• Experience designing and developing REST APIs and microservice-based architectures.
• Proficiency in at least one backend programming language such as:

o Python

o Go

o Rust

• Strong understanding of:

o Backend service architecture

o Distributed systems

o API design

o Asynchronous processing

o Concurrency

o Error handling and resiliency

• Experience with relational databases such as PostgreSQL, MySQL.
• Experience with database design, indexing, transactions, query optimization, and performance troubleshooting.
• Experience with caching and key-value technologies such as Redis.
• Experience with messaging or queuing technologies such as:

o AWS SQS

o RabbitMQ

o Kafka

o Redis-based queues

o Cloud-native messaging services

• Experience implementing authentication and authorization mechanisms such as OAuth 2.0, OIDC, JWT, and API keys.
• Experience implementing SSO (SAML based OIDC)
• Experience with observability tools and practices, including:

o Prometheus

o Grafana

o OpenTelemetry

o Distributed tracing

o Centralized logging

• Experience developing and managing background workers, task-processing systems, or asynchronous job pipelines.
• Experience with object or file storage systems such as Amazon S3, Azure Blob Storage, Google Cloud Storage, or equivalent technologies.
• Experience developing APIs and validating services using tools such as cURL, Postman, Bruno, or similar tools.
• Strong understanding of automated dev-testing, including:

o Unit testing

o API testing

• Experience with containerized application development using Docker.
• Familiarity with container orchestration platforms such as AWS ECS/Fargate or Kubernetes.
• Experience with Git, CI/CD pipelines, code reviews, and automated build/test workflows.
• Strong debugging, analytical, and problem-solving skills.

Preferred Qualifications· Experience designing backend systems that support machine learning or AI inference workloads.

• Experience with inference request orchestration, scheduling, queuing, batching, or worker management.
• Familiarity with model-serving or inference platforms such as:

o NVIDIA Triton Inference Server

o vLLM

o SGLang

o Similar model-serving frameworks

• Experience designing systems that schedule workloads across CPU and GPU compute resources.
• Familiarity with GPU resource management and inference worker scaling.
• Experience with asynchronous task processing frameworks.
• Familiarity with API gateways, service discovery, and service-mesh technologies.
• Experience with any cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Familiarity with infrastructure-as-code tools such as Terraform.
• Experience with performance and load-testing tools such as k6, Locust, JMeter, or similar frameworks.
• Experience building systems with high availability, horizontal scaling, and fault-tolerance requirements.

Key Technical SkillsBackend Development: Python, Java, Go, C#, or Rust

API Development: REST, HTTP, JSON, gRPC

Databases: PostgreSQL, MySQL, SQL Server, NoSQL databases

Caching & Messaging: AWS SQS, Redis, RabbitMQ, Kafka, task queues

Storage: Object storage, file storage, cloud storage services

Distributed Systems: Microservices, asynchronous processing, background workers, job scheduling

Testing: Unit testing, integration testing, API testing, cURL, Postman, Bruno

Version Control & Delivery: Git, CI/CD, code reviews, automated build/test workflows

Infrastructure: Docker, AWS ECS/Fargate, Kubernetes, CI/CD

Observability: Logging, metrics, monitoring, tracing

ML Infrastructure: Inference APIs, job scheduling, worker orchestration, model-serving integration

Retrieval & Inference: pgvector hybrid retrieval, SSE streaming, in-process ONNX inference, multi-model vision consensus on AWS Bedrock