LGR database consolidation will be difficult, but authorities are not powerless while decisions are still evolving, there is practical readiness work they can and should start now. Roisin Murray, Head of Onboarding & Transition at Idox, explains how to build the foundations for a successful transition to a single platform​.

Introduction

UKAuthorITy’s survey of key digital, data and technology (DDaT) issues affecting Local Government Reorganisation (LGR) underlined the importance of data: sources, quality and accessibility. It also raised concerns regarding awareness and understanding of the challenges ahead. Two thirds of authorities involved in the current round of reorganisation were confident they had clarity over the location of the data required by the new authority – yet more than half of those with LGR experience said it was not the case. Asked if they had access to all the data they would need from multiple line of business systems, three-quarters of those in the current round of reorganisation and all of those with experience replied ‘no’. 

Data readiness is a vital foundation for successful LGR. Authorities require not only data awareness, as highlighted by this survey, but also robust data cleansing and consistent data retention policies before data migration can be considered. With the right approach and supportive partner, however, there are clear steps authorities can take today to improve data readiness and minimise the data migration challenges.

Data awareness – locating diverse sources of data

LGR brings together councils running different systems, holding data in different formats and following different processes. While the objective of consolidating systems into a single platform is clear, the foundation for successful transition is good quality, accurate, consistent data. And this is complex. Even a single council will have not only multiple systems – often from different providers – but also a complex mix of integrations, using connectors and APIs to enable cross-function data sharing. Many authorities still have pockets of standalone data, including spreadsheets, that need to be included in any data migration process. Multiply that position by the number of joining authorities and the overall data picture is enormously complex.

An additional factor that will influence the data migration programme is whether each contributing authority is integrated with the Land Registry (HMLR). With land charges data highly dependent upon a range of data sources, including planning, building control and environmental records, HMLR integration status is hugely influential to the migration strategy. 

It is, therefore, important to undertake a data discovery process sooner rather than later. A detailed assessment of the applications, suppliers, systems, integrations, data ownership, workflows, reporting and regulatory variations across authorities is required to build a complete picture of the current landscape. Working with a transformation partner can be a vital step in helping to identify all the sources of data in use across contributing councils, leveraging expertise and understanding of systems, data and operational challenges to build a complete data overview across all authorities.

Data quality – enforcing cleansing and retention rules

In addition to clarifying the full extent of data resources across all joining councils, it is also essential to build data confidence by determining the quality of these diverse resources. Highlighting duplicate or conflicting records is important not only to improve the quality of the underlying data resource; it will also play a key role in reducing data volumes. And this is a vital consideration: cost reduction and on-going cost control are important objectives within the LGR process. Taking this opportunity to address and, where possible, reduce data volumes will both ease the migration process and minimise future data storage costs.

As such, defining clear data retention policies is also important. At Shropshire Council, which became a unitary in 2009, data migration was repeatedly identified as the most critical and highest risk element of LGR. Claire Richardson, Business Development Manager, at Shropshire Council is adamant that authorities need to avoid the ‘retain everything’ model that can create enormous, expensive data storage requirements.  “The biggest mistake would be thinking you can just lift and shift everything and put it in one big cupboard. Don’t do that. Take what you actually need,” she says. 

In addition to defining rigorous data retention models, the success of any data migration is predicated on data quality. As Shropshire Council’s Lisa Jones, Systems Development Officer, insists, data cleansing and standardisation are essential prior to migration. “Rubbish in means rubbish out, “she says. “If your document types, descriptions and codes aren’t right, the system is no use to anyone.” 

Data workshops – gaining clarity and focus

An experienced technology partner can facilitate LGR data readiness workshops, during which key questions can be asked. What are the essential data resources required for vesting day? What data is needed to support the new authority in the future? Where are the inconsistencies in data recording between joining authorities?

Advice from authorities that have undertaken LGR migration, including Somerset and Shropshire Councils, includes:

• Adopt a regulatory-first approach to data retention, with a focus on retaining the minimum data needed for statutory reports and outputs.
• Assess the wider use of data throughout the authority to ensure a complete understanding of data value.
• Use a simple coding structure to distinguish between internal and external consultees to streamline reporting.
• Leverage the Unique Property Reference Number (UPRN) to provide a consistent data resource across Built Environment service areas.
• Prioritise goals, such as consolidating property and street data, as these datasets are the backbone of many council services.

This process is key to defining robust data retention policies and instigating data cleansing projects that help authorities to reduce data volumes. It supports the identification of data migration priorities and the creation of a clear plan for the evolution from multiple authorities and platforms to a single, successful solution.

Data tools – leveraging technology to accelerate change

A range of tools can help accelerate data cleansing, including the use of AI to identify potential duplication, inconsistency and variation in how information is recorded. Pattern recognition can highlight where contributing councils use the same core system in different ways. For example, one authority may expect planning officers to record only basic information in planning notes, while another may require far greater detail. Identifying these variations early helps prevent hidden differences in working practice from being carried forward into the consolidated system. 

Highlighting differences early in the data assessment phase will ensure a consistent approach can be determined long before the consolidated system is deployed. Indeed, the data assurance process – including both cleansing and data retention policymaking – can be the foundation for the definition of future-state processes and practices. It provides a forum for reviewing variations between authorities and highlights opportunities for automation and efficiency within the consolidated authority. 

Tools are also playing an increasingly important role in accelerating the data migration process. Building on extensive experience of migrating authorities from on-premise systems to Idox Cloud, the latest evolution of Idox’s Extract, Transform, Validate and Load (ETvL) tool is being developed specifically to support more complex LGR data migrations. Utilising modern data manipulation techniques, proven baseline mappings and automated validation, ETvL streamlines the process of extracting data from multiple source systems, transforming it into a consistent structure, validating it against agreed rules and loading it into a single, unified Idox Cloud database.

Conclusion – getting data ready

Migrating to a single golden source of data is the end goal, yet with contributing authorities currently at very different stages of the unitary evolution, there is no single route forward. Indeed, many have yet to even consider the data readiness situation while other decisions are still under review.

It is, however, vital to start thinking about this as soon as possible. Data discovery. Data cleansing. Data retention policies. These are all projects that can be initiated today, irrespective of future plans or, indeed, timelines. The single platform objective can only be achieved with clear, accurate understanding of the current picture. It is the foundation for creating a robust implementation plan with target solution design, migration strategy, integrations, resourcing and training. 

An experienced partner with understanding not only of system and operational challenges but also the full potential of systems to drive new levels of efficiency, can facilitate workshops to create a clear vision of the future operating model. “[Data migration] is an opportunity to minimise the data, clean it up and consider how to simplify and speed up every single process,” concludes Shropshire Council’s Claire Richardson.

Get your data ready for LGR with support from Idox, click below to connect with our expert team.