UK jobs. Bigger possibilities.
148,000+ live jobsData Engineer (G7)
On-site in London·Added today (last checked 16 Oct)
41 open roles
The role
Details
Reference number
Salary
GBP
Job grade
Contract type
Business area
Type of role
Digital
Information Technology
Strategy
Working pattern
Number of jobs available
Location
Please note that Caxton House, London is only available to existing DWP employees already based at Caxton House and Sanctuary Buildings, London. All other locations are available to all candidates.
Please find further information on the Corporate hub locations here.
Please note that for candidates requesting to work in Newcastle, the role will initially be based at Benton Park View. It will then move to 1 Pilgrim Place in Newcastle city centre by the end of 2027.
For candidates requesting to work in Sheffield King's Court, please note these roles will initially be based at King's Court and then move to 1 West Bar by early 2028
About the job
Job summary
Please note this role requires you to pass Security Check clearance. For further information, please see 'Selection process details'.
You must have resided continuously in the UK for at least 3 of the last 5 years, 2 of which must have been the immediately preceding years from the point of applying for this job.
Are you a Senior Data Engineer with expert-level experience in Azure data engineering, ETL/ELT pipeline design and technical team leadership?
Join DWP as part of our Data Engineering community, where you will lead the design and delivery of secure, scalable data solutions that power services used by millions of people across the UK. This role plays a key part in ensuring our data estate is reliable, well-governed and continuously improving, enabling analysts, data scientists and operational teams to access trusted data at scale.
Job description
Roles and Responsibilities
- Lead the team in designing, developing and maintaining scalable ETL/ELT pipelines using a strong knowledge of PySpark and Python to ensure best practice and standards are met in optimizing, validating and standardising all team data workflows as well as batch processes.
- Lead collection of complex data requirements from a wide range of stakeholders, interpret the data and shape data sets to for Data Analysts to fulfil requirements in their creation of analysis and insight dashboards.
- Lead the build and management as well as the supply of data to multiple systems and applications including database management. This to secure data availability and the construction of efficient scripts to extract data for cross business usage.
- Leading immediate team though large-scale data and platform transformation and/or migration programmes, ensuring best practice and building capability within the team as the transformation occurs. Set direction of change and proactively identify key issues.
- Apply a deep understanding and experience of cloud data engineering toolsets in developing complex pipelines, lead development of products and automation processes. Lead development of logging, monitoring and assuring tool performance information.
- Oversee, support and manage team in developing and using a range of operational tracking and cataloguing tools to track data and metadata through its journey such as: Google Cloud Data Catalog, AWS Glue Data Catalog, Azure Data Catalog, Informatica Enterprise Data Catalog, Alation, Apache Atlas etc. Ensuring compliance with business standards.
- Lead the use of Agile practices in the team to leverage data engineering tools like version control, automation, monitoring to streamline data pipelines, automate testing, deployment and track changes.
Skills England is an executive agency of the Department for Work and Pensions (DWP), working with employers, training providers, local partners and government to create better skills for better jobs. We play a leading role in identifying current and future skills needs across the economy, helping to ensure that people can access high-quality education, training and employment opportunities while supporting businesses to develop the skilled workforce they need. As part of the Digital and Data Services team you will play a key role in ensuring Skills England creates, maintains and uses data efficiently and effectively to drive decisions and produce high quality technical education products and labour market insights.
Locations for this team are: London, Coventry, Darlington, Sheffield, Leeds. Roles within other teams will be based in the locations advertised above. This can be discussed further, should you be successful.
Person specification
See selection process for further details.
If you would like to learn more about the role, please contact Aisha.gultasab@dwp.gov.uk.
Technical skills
We'll assess you against these technical skills during the selection process:
- Leading a team in designing Data Engineering Pipelines
Benefits
We also have a broad benefits package built around your work-life balance which includes:
- Working patterns to support work/life balance such as job sharing, term-time working, flexi-time and compressed hours.
- Generous annual leave - at least 25 days on entry, increasing up to 30 days over time (pro-rata for part time employees), plus 9 days public and privilege leave.
- Support for financial wellbeing, including interest-free season ticket loans for travel, a cycle to work scheme and an employee discount scheme.
- Health and wellbeing support including our Employee Assistance Programme for specialist advice and counselling and the opportunity to join HASSRA a first-class programme of competitions, activities and benefits for its members (subscription payable monthly).
- Family friendly policies including enhanced maternity and shared parental leave pay after 1 year's continuous service.
- Funded learning and development to support progress in your role and career. This includes industry recognised qualifications and accreditations, coaching, mentoring and talent development programmes.
- An inclusive and diverse environment with opportunities to join professional and interpersonal networks including Women's Network, National Race Network, National Disability Network (THRIVE) and many more.
Hybrid Working
This role may be suitable for hybrid working, which is where an employee works part of the week in their DWP office and part of the week from home. This is a voluntary, non-contractual arrangement and your office will be your contractual place of work.
If a hybrid working arrangement is suitable for the role and for you, you will normally be required to spend a minimum of 60% of your contracted working hours from your DWP office.
If you have a disability, caring responsibilities, or other circumstances that may affect your ability to meet the minimum office attendance requirement, please discuss this with us using the contact details in this advert.
Salary Information
Pay for this role is from £57,946 to £75,664.
The maximum salary for the grade is £68,205, however a Digital Allowance of up to £7,459 per annum is available for exceptional candidates. Digital Allowance is non-salary, and non-pensionable, and is classed as a temporary allowance. It is reviewed annually where it could be retained, amended, or removed.
Pay for this role in London is from £63,075 to £74,005. (Please refer to location details to check eligibility)
Our offer to successful candidates will be based on an assessment of your skills and experience as demonstrated at interview.
Existing Civil Servants who secure a new role on lateral transfer will maintain their current substantive salary.
Civil Servants who gain promotion will move to the bottom of the grade pay scale or receive a 10% increase applied to their current substantive salary, whichever is greater.
Any temporary allowances that you are currently in receipt of will not form part of the calculation to determine your pay. Any allowances that are in payment will cease when you move into your new role.
Things you need to know
Artificial intelligence
Selection process details
Stage 1: Application
Your application will consist of four parts:
1. A Personal Details application form.
2. Employment history - this should contain your work experience and any skills, qualifications and accomplishments relevant to the jobs you have completed.
3. Personal statement - up to 1000 words. This statement should be used to provide examples of how you meet the essential criteria below:
- PRACTITIONER: Ability to lead the team in developing complex ETL pipelines, including effective delegation and prioritisation of work.
- PRACTITIONER: Sound ability in communicating between the technical and non-technical: you can break down complex concepts to communicate them in plain language, adapting communication style to large audiences of none-specialists and senior stakeholders.
- PRACTITIONER: Strong understanding of the principles of data processing in a distributed or cloud systems, and ability to use this understanding to ensure and design robust coding in a distributed or cloud environment using a variety of native toolsets.
- PRACTITIONER: Understand how to collate and review complex technical requirements and specifications, leading the definition of test conditions for the team.
- PRACTITIONER: Understand how to lead and investigate complex problems/issues in data, pipelines and processes, systems and platforms, with an understanding of the level of a problem, for example, strategic, tactical or operational
- PRACTITIONER: Strong knowledge of assessing, mapping, and migrating complex data from a range legacy or on-prem systems to cloud-native formats and platforms across an area.
4. Technical statement (up to 250 words). The following statement is aligned to the required technical skill of: Leading a team in designing Data Engineering Pipelines. This statement should be used to provide example(s) of how you meet the criterion below:
- EXPERT: Strong understanding of leading the design and automation of ETL pipelines using PySpark and Spark SQL, ensuring seamless extraction, transformation, and loading of structured and unstructured data across cloud environments such as AWS, GCP and Azure.
Ensure that all examples provided in your statements are taken directly from your own experience and that you describe the examples in your own words. If you choose to use gen-AI to support your statements, you must follow the guidelines outlined in the Artificial intelligence and recruitment guide.
The sift panel will use the information in your employment history, personal statement, and technical statement to assess your experience, skills and knowledge.
An initial sift will be conducted using the technical statement. Candidates who pass the initial sift will be progressed to a full sift. If you do progress to a full sift, you will be provided with one combined overall assessment score for both your employment history and Personal Statement.
For Hints and Tips on completing your application visit Applying for jobs at DWP Digital.
Important Information
- You will be asked to complete your employment history. Any information that you would customarily share on a CV should therefore be entered onto the application form.
- Personal details that could be used to identify you including your name, contact details and address must be removed for your application to be considered.
- If your employment history, personal statement or technical statement contain any personal details your application will be withdrawn.
Stage 2: Interview 1
If you're successful at sift stage you will be invited to a video interview via Microsoft Teams.
You will be asked questions to further assess the experience and evidence you provided in your sift application.
Stage 2: Interview 2
If you are successful at interview 1, you will be invited to a video interview via Microsoft Teams. There, you will be assessed against the experience criteria listed under both the essential criteria and technical skill.
Interviews will take place from late November 2026 onwards. Sift and Interview dates to be confirmed.
Other information
All successful candidates and those placed on reserve will be posted in merit list order by location.
Please note that for these vacancies we recommend that applicants seek specialist advice on their eligibility for Visa Sponsorship (if applicable) when considering whether to apply for any role. We can never guarantee sponsorship, requests will be considered in line with the relevant DWP guidance and Home Office policy in place at the time.
Security Clearance Requirement
You must meet the security requirements before you can be appointed. The level of security needed is security check.
You must have resided continuously in the UK for at least 3 of the last 5 years, 2 of which must have been the immediately preceding years from the point of applying for this job.
For further information on National Security Vetting please visit the Demystifying Vetting website.
Further Information
At the Department for Work and Pensions (DWP) we value diversity and inclusion and actively encourage and welcome applications from everyone, including those that are underrepresented in our workforce.
Find out more about working for DWP on the DWP Careers Hub.
Reserve List
A reserve list may be held for a period of 6 months from which further appointments can be made.
If you are placed on a reserve list but we cannot immediately offer you a post, please note:
- If you are later offered the role you applied for, in a location you have expressed a preference for, and you decline the offer or are unable to take up the post within a reasonable timeframe you will be withdrawn from the campaign and removed from the reserve list, other than in exceptional circumstances (e.g. a contractual Part Time Working Pattern cannot be accommodated in the initial role offered or in cases of serious ill health).
- If DWP makes an offer of an alternative role or location to that which you originally applied for, and you decline that offer, you will be able to remain on the reserve list.
Disability Confident Scheme
If high application volumes are received, the benchmark for candidates to proceed to the next stage may be raised. In line with our commitment to the Disability Confident Scheme (DCS), we aim to advance all candidates applying under the DCS who meet the minimum standard. However, we may only progress those candidates who best meet the required standards.
Reasonable Adjustments
We consider visible and non-visible disabilities, neurodiversity or learning differences, chronic medical conditions, or mental ill health. Examples include dyslexia, epilepsy, autism, chronic fatigue, or schizophrenia.
If you need a change to be made so that you can make your application, you should contact the Government Recruitment Service via DWPRecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.
Complete the "Reasonable Adjustments" section in the "Additional requirements" page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you're deaf, a Language Service Professional.
If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'Contact point for applicants' section.
Integrity, plagiarism and Civil Service Principles
The Civil Service values honesty and integrity and expects all candidates to abide by these principles.
Please ensure that all examples provided in your application are taken directly from your own experience and that you describe the examples in your own words.
Examples of plagiarism can include:
- Presenting the work, ideas and experience of others as your own
- Copying content or answers from an online or published source that is not your own
Disclosure and Barring Service and Internal Fraud Database Checks
To process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service on your behalf. However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading.
Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant's details held on the IFD will be refused employment. A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government.
If successful and transferring from another Government Department, a criminal record check will be carried out.
Important
New entrants are expected to join on the minimum of the pay band.
Before applying for this vacancy, current employees of DWP should check whether a successful application would result in changes to their terms & conditions of employment, e.g. mobility, pay, allowances. If you are a current employee and are successful you must be able to be released from your current post within four weeks.
Those on protected TUPE/ COSoP terms and conditions applying on promotion or voluntary permanent level move will adopt DWP's Terms and Conditions and this may have a different impact on pay and allowances. Please review this prior to acceptance of a role.
Civil Servants that would transfer into DWP from other government organisations, following successful application, will assume DWP's terms & conditions of employment current on the day they are posted, unless DWP has stated otherwise in writing. Accepting a post will be taken to mean acceptance of revised terms & conditions.
Civil Servants applying on promotion will usually be appointed to the salary minimum or within 10% of existing salary.
Any move to DWP from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may, however, be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility at the Childcare Choices Website.
Feedback
Feedback will only be provided if you attend an interview or assessment.
Security
See our vetting charter (opens in a new window).
Nationality requirements
This job is broadly open to the following groups:
- UK nationals
- nationals of the Republic of Ireland
- nationals of Commonwealth countries who have the right to work in the UK
- nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
- nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
- individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
- Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service
Working for the Civil Service
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
Diversity and Inclusion
Apply and further information
Contact point for applicants
Job contact :
- Name : Aisha Gultasab
- Email : Aisha.gultasab@dwp.gov.uk
Recruitment team
- Email : digitalrecruitment.grs@cabinetoffice.gov.uk
Further information
If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission. Click here to visit the Civil Service mmission.
About Department for Work and Pensions (DWP)
- Industry
- Government Administration
- Website
- gov.uk
More jobs like this
or browse Mid-level·Civil Service & Government·Government Administration