LLMOps Engineer

Every day, millions of shipments move around the world on data that nobody is looking at in real time. Orders, ASNs, invoices, and tracking events flow between 3PLs, forwarders, brands, and retailers, then get flattened into a status that tells you something went wrong only after it already has.

SwiftB2B exists to change that. We are an AI operating system for global supply chains. One platform where 3PLs, freight forwarders, brands, retailers, and manufacturers can see every item, shipment, and exception as it happens. The platform does not stop at flagging a delay; tt works out what the delay costs, weighs the options, and starts the fix.

We built it on 15 years of SwiftAnt’s work in supply chain and EDI technology, and on a team that has spent careers in logistics, retail, and enterprise software. We are a Databricks partner and the platform runs on the lakehouse.

Our product has a dozen LLM-backed features and the number is growing: natural-language query, exception narratives, agents that draft and act, document extraction over customs paperwork and rate cards. Every one of them needs a model behind it, a prompt that is versioned, an eval that runs before release, a cost per call somebody is watching, and a way to roll back when a provider changes a model under us. Right now that is spread across the people who built each feature, and we are looking for one person to own it.

What you will do

• Own the serving layer: Databricks Model Serving, Foundation Model APIs, direct provider APIs (Anthropic, Azure OpenAI), and the gateway in front of them. Routing, fallback, rate limits, key rotation.
• Version prompts, tools, and agent configurations like code. Nothing reaches production without a diff, a review, and an eval run.
• Build and run the evaluation pipeline: regression sets per feature, LLM-as-judge where it is honest, human review where it is not. Gate releases on the results and publish them.
• Put a number on everything: tokens, cost per feature per customer, latency percentiles, cache hit rates, escalation and refusal rates. Build the dashboard the CTO looks at.
• Tracing and observability with MLflow and OpenTelemetry. When a customer asks why the system said something, we can replay it.
• Handle model lifecycle: provider deprecations, model upgrades tested against the eval set before switching, fine-tuned or distilled models where they beat the frontier on cost.
• Security and data handling for LLM traffic: PII redaction, prompt-injection defences, data residency per customer, audit logs. Work with the security engineer on the controls and with the quality team on the evidence.
• Write the runbooks. An LLM feature that only its author can operate is a risk nobody priced.

What we are looking for

• Two or more years running ML or LLM systems in production, with responsibility for uptime, cost, or quality.
• Hands-on with MLflow, Databricks Model Serving or an equivalent serving stack, and at least one LLM gateway or router.
• Strong Python, solid CI/CD, Docker, and infrastructure as code on Azure.
• You have built an eval pipeline and used it to block a release.
• You understand LLM cost and latency well enough to cut both without breaking quality, and you have done it.
• Experience with tracing and observability tooling for LLM apps (MLflow Tracing, Langfuse, Arize, or similar).
• Familiarity with prompt-injection and data-leak risks and the practical defences.
• Databricks certifications are a plus. Supply chain background is an advantage.
• A degree in Computer Science or a related field, or equivalent practical experience.

Not sure you tick every box?

Apply anyway. Nobody we have hired matched the list line for line. If you have kept an LLM feature running when the provider changed something underneath you, we want to talk to you. Tell us about the eval that saved you, or the one you wish you had. That tells us more than a polished CV!

Equal opportunity

SwiftB2B hires on talent, skill and curiosity. We do not discriminate on the basis of race, religion, nationality, gender, sexual orientation, age, disability, or any other protected characteristic. If you need an adjustment at any stage of the process, tell us and we will make it happen.