UK jobs. Bigger possibilities.

Senior Data Engineer

Hybrid in London·Added yesterday (last checked 7 Nov)

Our summary

Salary
£68,332 - £84,826 a year

About £4,180 - £4,980 a month after tax.

Work mode
Hybrid, London
Experience
Senior level
Pension
28.97% employer pension
Commute
Cycle to work scheme

Skills

Requirements

What we're looking for

It is essential that you have:

  • Expert Python and SQL skills: design and build reusable software, performance optimisation, automated testing, and troubleshooting complex production systems (Lead Criteria)
  • Experience leading teams to develop resilient Python data pipelines, including testing, automation, performance tuning and troubleshooting
  • Proven ability to lead the design and delivery of secure, resilient AWS infrastructure, applying effective networking, access control, monitoring and secrets management through Infrastructure as Code using Terraform
  • Experience building Docker and container-based workflows
  • Broad experience developing and maintaining enterprise-level data solutions
  • Experience defining and implementing data quality and automated testing strategies, including validation, monitoring and controls across complex data pipelines

Nice to have

It is desirable that you have:

  • Experience of line management including setting objectives, managing performance and coaching and developing others
  • Experience using agile methodologies on data projects

Pay & benefits

What you'll get

£64,332 - £84,826

London: £68,332 to £84,826 / National: £64,332 - £80,826 (including allowance). Your salary will be determined by your skills and capability as assessed at interview.

If you join us, you will get:

  • Learning and development tailored to your role
  • A flexible, hybrid working environment with options like condensed hours
  • A culture encouraging inclusion and diversity
  • A Civil Service pension with an average employer contribution of 28.97%
  • Annual leave starting at 25 days rising to 30 days with service
  • Three paid volunteering days a year
  • An employee benefits programme including cycle to work

Visa & relocation

This job is broadly open to the following groups:

  • UK nationals
  • nationals of the Republic of Ireland
  • nationals of Commonwealth countries who have the right to work in the UK
  • nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
  • nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
  • individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
  • Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

The role

The Department for Business, Innovation, Science and Trade (BIST) helps businesses invest, innovate, export and grow across the UK. The Digital, Data and Technology (DDaT) directorate helps to deliver this by designing, building and running the services, platforms and technology that businesses and colleagues rely on every day.

Our teams work on a wide range of services and technologies, including:

  • digital services that help businesses access government support, guidance and opportunities
  • data and AI products that support decision making and enable innovation
  • internal tools and services that help colleagues work more effectively
  • secure technology and platforms that support critical government services
  • cyber security services that protect systems, data and users

By joining DDaT, you'll work on services used by businesses, colleagues and citizens across the UK. You'll be part of multidisciplinary teams focused on improving services, solving complex problems and delivering better outcomes for users.

Data engineers at BIST are pivotal to our mission of driving economic growth by placing data at the heart of decision-making. We leverage leading cloud and open-source technology to build scalable, business-critical data pipelines, products and services, ensuring our users-whether within BIST, across government, or the public- have access to the right data in the optimal format.

This could include:

Using Apache Airflow and various AWS tools to build robust, scalable pipelines

Using Terraform to deploy and maintain our Postgres-backed data platform

Our cross-functional, agile teams work collaboratively on a range of projects from implementing AI tools, to our data science and analytics platform, to our system for managing the British Industrial Competitiveness Scheme (BICs), and beyond. Data Engineers also belong to a vibrant community (both within DBT and across government) who share and promote best practice.

We are seeking a collaborative and innovative problem-solver to help us deliver better outcomes for our users. You will use your expertise in SQL, Python and AWS to build robust data products, develop scalable data products. You will also have experience in supporting and developing more junior members of the team.

What you'll do

You will:

  • Lead the design, build, and testing of data products: Develop robust data products by integrating feeds from multiple systems, utilising a variety of storage technologies and access methods
  • Ensure the delivery of resilient and scalable solutions: Select and deploy the most suitable technologies to create data solutions that are resilient, scalable, and future proof
  • Have problem-solving expertise: Understanding of common issues in databases, data processes, products, and services, and apply standard solutions effectively
  • Collaborate with stakeholders: Work closely with a diverse range of stakeholders to understand their requirements, establish project goals, and define deliverables.
  • Support users and solve issues: Assist users in utilising data tools, resolve issues promptly, and escalate when necessary
  • Promote best practices: Develop and advocate for best practices within the team.
  • Encourage team building and development: Organise initiatives to foster team building and professional development
  • Provide line management and support to junior team members, guiding their growth and development
  • Drive continuous improvement: Stay informed about the latest trends and best practices in data engineering to drive continuous improvement

How you'll work

Flexible working, Full-time, Part-time

Reference number

489204

Job grade

Grade 7

Contract type

Permanent

Business area

DBT - CS - Digital, Data and Technology

Type of role

Administration / Corporate Support
Architecture and Data
Digital
Information Technology
Project Delivery

Number of jobs available

1

Location

Belfast, Birmingham, Cardiff, Darlington, Edinburgh, London, Manchester

Behaviours

We'll assess you against these behaviours during the selection process:

  • Leadership
  • Communicating and Influencing
  • Delivering at Pace

Technical skills

We'll assess you against these technical skills during the selection process:

  • Data development process
  • Data innovation
  • Data modelling
  • Problem management
  • Programming and build (data and analytics engineering)

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

How we offer

Offers will be made in merit order based on location preferences. If you pass the bar at interview but are not the highest scoring you will be held on a 12-month reserve list in case a role becomes available. If you are judged a near miss at interview, you may be offered a post at the grade below the one you applied for.

This role requires SC clearance. BIST's requirement for SC clearance is to have been present in the UK for at least 3 of the last 5 years. Failure to meet this requirement will result in your application being rejected and your offer will be withdrawn.

Checks will also be made against:

  • departmental or company records (personnel files, staff reports, sick leave reports and security records)
  • UK criminal records covering both spent and unspent criminal records
  • your credit and financial history with a credit reference agency
  • security services record
  • location details

Security

Successful candidates must undergo a basic (or equivalent) criminal record check.

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).

People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Apply and further information

This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.

The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).

Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Job contact

  • Name : DDaT Recruitment
  • Email : Ddat.recruitment@businessandtrade.gov.uk

Further information

Our recruitment process is underpinned by the principle of appointment on the basis of fair and open competition and appointment on merit, as outlined in the Civil Service Commissioners' Recruitment Principles. If you feel your application has not been treated in accordance with these principles and you wish to make a complaint, you should in the first instance contact DBT by email at Resourcing@trade.gov.uk. If you are not satisfied with the response you receive, you can contact the Civil Service Commission, which regulates all Civil Service recruitment. For further information on bringing a complaint to the Civil Service Commission please visit their web pages: Civil Service Commission Complaints

Share this page

Hiring process

Apply before Midday on Friday 27th November 2026

As part of the application process you will be asked to upload a two-page CV and complete a 750-word personal statement outlining how you meet the essential skills and experience listed above. You can use bullet points and subheadings if you prefer.

Sift will be from week commencing 30th November 2026

Interviews will be from week commencing 14th December 2026

Please note these dates are indicative and may be subject to change.

If there is a high volume of applications, we will sift looking at the lead criteria 'Expert Python and SQL skills: design and build reusable software, performance optimisation, automated testing, and troubleshooting complex production systems' only. You may then be progressed to full sift or straight to interview.

At the interview stage for this role, you will be asked to demonstrate relevant Technical Skills and Behaviours from the Success Profiles framework. These are role specific and in line with the DDaT Capability Framework

You will also be asked to undergo an assessment and will be informed on the topic on the day.

About Department for Business and Trade (DBT)

Industry
Government Administration

In their own words

About the company & team

We are committed to creating an inclusive and supportive workplace. In recognition of this, we were named Best Public Sector Employer at the Women in Tech Employer Awards 2025!

This role can only be worked from within the UK, not overseas. If you are based in London, you will receive London weighting. BIST employees work in a hybrid pattern, spending 2-3 days a week (pro rata) in the office on average. Travel to your primary office location will not be paid for by BIST, but costs for travel to an office which is not your main location will be covered.

You can find out more about our office locations, how we calculate salaries, our diversity statement and reasonable adjustments, the Recruitment Principles, the Civil Service code and our complaints procedure in the candidate pack attached to this advert.

Find out more about life at BIST, our benefits and meet the team by watching our video or reading our blog!

Feedback will only be provided if you attend an interview or assessment.

  • Email : Ddat.recruitment@businessandtrade.gov.uk

Additional information

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).

The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

All open roles at Department for Business and Trade (DBT)
WhatsApp