Senior Data Engineer

Job Description

Our client is a European industrial group building a group-wide manufacturing data platform. Plant and IoT telemetry streams through Kafka into a Databricks lakehouse, where Python pipelines and modeled transformations serve analytics and ML consumers across the group. You join the client's data platform team full-time. You own pipelines end to end, from Kafka ingestion through to the data contracts and quality gates that downstream teams build against. You work under the client's direction, alongside their platform architects, analytics engineers, and the ML team that consumes your tables.

What you'll be doing

• Build and operate Kafka ingestion for plant and IoT telemetry from the client's manufacturing sites
• Design PySpark pipelines on Databricks that move telemetry from raw ingestion to curated Delta tables
• Model transformations in dbt and Spark SQL, turning raw plant data into documented, versioned datasets
• Own data contracts between plant-side producers and the analytics and ML teams consuming the platform
• Ship schema validation, freshness and volume checks that stop bad data before consumers see it
• Tune Spark jobs and Delta tables for cost and latency as plant coverage grows
• Run the platform day to day: monitor pipelines, resolve incidents, manage schema migrations and backfills
• Review pipeline changes from engineers across the group and harden the shared ingestion libraries

What you'll need

• 6+ years of data engineering, with 3+ years running Spark pipelines in production
• Strong Python engineering skills, with pipelines shipped as tested and reviewed code
• Production Kafka experience: topic design, consumer groups, and schema management
• Hands-on experience with Databricks or an equivalent Spark lakehouse stack, including Delta Lake or a comparable table format
• Solid SQL and data modeling, with dbt or a comparable transformation framework
• Experience defining data contracts and quality gates for analytics and ML consumers
• Comfort operating what you build: pipeline CI/CD, monitoring, and incident response
• Professional working proficiency in English
• Based in the EU with working hours overlapping CET

Nice to have

• Exposure to manufacturing data sources such as OPC UA, MES, or plant historian systems
• Databricks Certified Data Engineer Professional or an equivalent Spark certification
• Terraform and Kubernetes experience for pipeline infrastructure
• German at B2 or above
• Availability for occasional business travel within the EU

Engagement terms

• Remote-first. Deliverable-based, no time tracking.
• Monthly wellness allowance, scaling with tenure.
• Annual learning budget, scaling with tenure.
• Home office setup allowance, refreshed every two years.
• 25 days annual leave plus one additional day per year of tenure.
• Birthday off.
• Family leave and private healthcare coverage.