Software / Data Engineer - Compliance Assurance & Streaming Data

Software / Data Engineer - Compliance Assurance & Streaming Data Location: Chester (Hybrid)A leading global financial institution is seeking a Senior Data Platform Engineer to help build and enhance a strategic enterprise platform responsible for ingesting, processing and assuring large-scale communication, collaboration and transactional data. We're looking for an engineer who can own the entire data lifecycle, from source-system integration and streaming ingestion through to transformation, storage and the delivery of production-grade compliance and assurance controls. You'll work at the intersection of software engineering, data engineering and regulatory technology, building scalable solutions used across a highly regulated environment. The role offers exposure to modern data-platform technologies, complex distributed systems, enterprise-scale datasets and emerging AI-enabled engineering practices. What You'll Be Doing End-to-End Platform EngineeringDesign and develop resilient enterprise-scale data pipelinesBuild integrations across vendor APIs, streaming technologies and distributed data platformsEngineer high-volume batch and near real-time processing solutionsDevelop reusable ingestion and processing frameworks rather than one-off solutionsImplement robust data handling patterns including replay, backfill, recoverability and schema evolutionDefine and enhance operational telemetry, monitoring and platform observabilityStreaming & Distributed Data ProcessingBuild and optimise solutions using:Apache KafkaApache NiFiApache Spark / PySparkHadoop ecosystem technologiesHive and large-scale data storage platformsDevelop scalable matching, enrichment and reconciliation servicesCreate canonical data models spanning multiple source systemsEnsure data lineage, quality and governance requirements are metControl & Assurance EngineeringTranslate regulatory and operational requirements into deterministic control logicDevelop reconciliation, completeness, configuration and assurance controlsProduce auditable outputs, exception reporting and evidence frameworksBuild monitoring and alerting capabilities for compliance-critical processesWork closely with engineering, architecture, risk and operational stakeholdersSoftware Engineering & DeliveryWrite clean, maintainable and testable Python and PySpark codeContribute to CI/CD pipelines and automated deployment processesParticipate in code reviews and engineering standards initiativesDeliver solutions through Development, UAT and Production environmentsSupport troubleshooting and root-cause analysis across distributed systemsRequired Experience We're interested in engineers who can demonstrate strong expertise across several of the following areas: Core Technical SkillsPython development in enterprise environmentsApache Spark / PySparkApache KafkaApache NiFiHadoop and HiveAdvanced SQLREST API integration and developmentData modelling and data lineageDistributed systems engineeringCI/CD and modern software delivery practicesGit and collaborative development workflowsAdditional Desirable ExperienceMicrosoft AzureMicrosoft GraphEntra IDService principals and secure identity patternsGrafana, Splunk, Tableau or Power BIJava or another strongly typed languageAI-assisted development tools including GitHub CopilotExposure to agentic AI, orchestration frameworks or automation toolingIdeal Background Candidates are likely to come from:Financial servicesBanking and capital marketsRegTechLarge-scale enterprise data engineering environmentsTelecommunications or communications platformsHighly regulated industriesExperience working with voice, communications or collaboration technologies would be advantageous, as would any exposure to assurance, reconciliation or control-focused platforms. What Success Looks LikeNew data sources are onboarded through reusable engineering patternsPipelines are resilient, observable and fully recoverableControl logic is transparent, auditable and trustedData quality and lineage are clearly understoodSolutions are delivered through a disciplined SDLC into production environmentsThe platform scales to support new technologies, controls and business requirements