Digitalization of the electricity infrastructure: a key enabler for the decarbonization and decentralization of the power sector

The digitalization of the electricity infrastructure is transforming the power industry and enabling its decarbonization and decentralization. In the electricity sector, digitalization is not a novelty but a process that has come in successive waves and whose implications go beyond technological change. Digitalization started at the level of transmission networks and large generation assets. By increasing the availability and usability of data on the status of the electricity network, it enabled more efficient and secure system operation. It also supported the creation of competitive wholesale markets and their integration at the regional level. More recently, digitalization has expanded its outreach to cover distribution networks and consumers’ premises, allowing a reduction in costs and an improvement in quality of service. It has also laid the basis for the establishment of innovative retail markets which empower final customers and provide opportunities for new entrants, while challenging existing approaches to sector regulation.


Introduction *
Digitalization is deeply transforming the electricity industry. The expanding application of Information and Communication Technologies (ICT) to the whole electricity infrastructure, from power plants to transmission grids and from distribution networks to end-use appliances, makes available an increasing amount of data on the status of the electricity system and enables their use to take decisions about the production, transmission, distribution and consumption of energy. 1 The availability and usability of data allow not only significant cost reductions but also the emergence of new forms of interaction between the actors involved in the sector and the development of new business models which might have disruptive implications for the industry's organization and regulation (WEF and Accenture 2016;IEA, 2017;Glachant and Rossetto, 2018;Brown et al., 2019;CEER, 2019b).
In the 1970s, vertically integrated electric utilities were among the pioneers in the application of ICT to their activities and assets. Later, digital technologies played a key role in the creation of wholesale markets for power and the operation of an unbundled electricity system that in various parts of the world was also supporting regional integration. Since the early 2000s, digital technologies have been penetrating the distribution grid and entering the premises of final customers. The deployment of smart meters and the debate about smart grids are examples of this digitalization wave, which now seems to be followed by a new one, where peer-to-peer (P2P) energy trading, energy communities and transactive energy models look like the new buzzwords. It is too early to judge about the success of these new concepts, but their potential for disruption seems to be high.
Digitalization is a major driver of change in the electricity industry nowadays, but it is not the only one. Decarbonization and decentralization are two other important trends that together with digitalization form the so-called three Ds. Decarbonization refers to the gradual removal of fossil fuels from the generation mix and the increasing reliance on other energy sources whose use does not emit carbon. This shift reflects the strong policy support received mostly by renewable energy sources (RES) in the past few years to address climate change and the subsequent cost decline in the related generation technologies. On the contrary, decentralization describes the multiplication of actors within the system able to choose autonomously where and when to produce electricity. 2 Started in many industrialised countries around the end of the twentieth century with the dismantle of legal monopolies and the entry of new market players, decentralization has been accelerating in the past few years due to the uptake of distributed generation (DG), 3 frequently but not always RES-based, and the appearance of a growing number of prosumers and prosumagers, both individual and collective. 4 The three Ds are strongly interconnected and affect one another. In this regard, digitalization represents a fundamental enabling factor of both the decarbonization and decentralization of electricity (Sivaram, 2018;Tagliapietra and Zachmann, 2018). The integration of a growing share of RESin particular if variable in nature like solar and windand the effective coordination of a multitude of medium to small-sized active actors connected to the public grid would be more expensive, if at all possible, without the solutions that digitalization offers. At the same time, decarbonization and * decentralization pull further investments and advances in the digitalization of the electricity sector. The rapid penetration, for instance, of small-scale photovoltaic (PV) units connected to distribution grids that followed the introduction of generous feed-in tariffs created a series of challenges for the efficient, secure and continuous delivery of electricity to network users. This development has induced distribution companies to accelerate the digitalization of their networks and has incentivized some market actors to develop innovative offers that frequently rely on digital applications.
The combined action of the three Ds is also blurring the traditional boundaries of the electricity sector. The development of new sources of demand-side flexibility together with the possibility to electrify various final energy uses and operate in a smart and automated way a myriad of interconnected energy-consuming appliances, call for and at the same time enable a convergence of different supply chains and industries, notably heating and cooling, transport, and telecommunications. This tendency, empowered by advances in digitalization and already noted by several companies both within and outside the electricity industry, is becoming an important pillar of recent energy policy developments, as exemplified by the EU Strategy for Energy System Integration published in July 2020 (EC, 2020).
In this chapter, we explore the digitalization of the electricity infrastructure and its transformative implications for the various segments of the electricity supply chain, from generation to retail, while passing through grids and system operation. By illustrating what digitalization means for the sector, the chapter aims to provide an overview that could be particularly interesting for those readers who come from outside the industry or who entered it recently. Being aware that electricity systems around the world are quite different in terms of structure and organization, we focus here on those systems that have been subject to restructuring and liberalization, as it is typically the case in Europe, North America, Australia and New Zealand. In Section 2 we first describe what the technologies employed to generate, transmit, analyse and use data are, and then provide a concise overview of the main types of data that are present nowadays in the electricity sector. Sections 3 and 4 consider the impact of digitalization on the way physical assets are planned and maintained, as well as operated. Section 5 looks at the possibilities enabled by digitalization to coordinate the different autonomous actors operating in the interconnected electricity system. The emergence of new products that empower final customers and open the door to the entry of new players in the sector is investigated in Section 6, together with the implications that these changes have on the traditional managers of the electricity infrastructure. Section 7 examines the challenges posed by digitalization to the organization and regulation of the sector. Finally, Section 8 concludes the chapter by summarising the main considerations that emerged.

Digital technologies and the data layer
A comprehensive data layer has emerged over time on top of the physical one through the application of an evolving set of digital technologies including sensors, control devices, communication networks and software applications. This data layer integrates different domains of the electricity sector, from large generating assets and networks to markets and consumers.
In this section, we set the basis for the rest of the chapter by providing an overview of the digital technologies adopted in the electricity sector and the data layer that they create. We do not pretend to be exhaustive here; on the contrary, our aim is to clarify a number of terms that are used throughout this chapter and provide the reader with a broad overview. 5 Section 2.1 describes a selection of the most relevant and representative technologies, based on the fundamental functions they perform. Section 2.2 focuses on the data layer and provides a classification of the types of data that can be found in the digitalized electricity sector. 5 The interested reader may find a comprehensive description of the technologies and data layer in Vadari (2018). An excellent and up-to-date description of digital technologies applied to the electricity sector is also available on https://www.entsoe.eu/Technopedia/. European University Institute 3

Overview of the digital technologies adopted in the electricity sector
A broad number of digital technologies are applied to the electricity sector nowadays. Consistent with their nature of multi-purpose technologies (see Chapter 2 by Knieps in this book), they are usually not specific to this sector, although they may have been adapted to satisfy its specific needs and requirements. Broadly speaking digital technologies are employed in electricity to fulfil a series of functions that from a conceptual point of view can be classified as: 1) generation (collection) of data, 2) transmission of data, 3) analysis of data, and 4) control implementation. 6 The function performed by each technology can be used as a classification criterion, keeping in mind, however, that in practical terms a given technology may well perform more than one function at the time and its inclusion in a certain category rather than another one may present some degree of arbitrariness.

Technology to generate data
Different types of sensors and metering devices are applied to various elements of the electricity networks and the connected generation and consumption units in order to generate data about them and enable their monitoring. Sensors and metering devices measure a plurality of variables. These include, but are not limited to, the voltage level on a line, the power withdrawn from or injected into the grid at a connection point, the air temperature or the wind speed at a certain site and so on. The constant decline in the cost of sensors and their improved sensitivity and connectivity have justified their increasing use and combination in sophisticated systems like the supervisory control and data acquisition (SCADA) systems that transmission system operators (TSOs) and the operators of large power plants have put in place since the 1990s (IEA, 2017). In SCADA systems, remote terminal units are located in substations or power plants to collect a series of data on the status of the circuit breakers, as well as the voltage, current and power levels. Every few seconds these data are reported to a central computer that estimates the situation in the grid or the power plant based on the inputs received (MIT, 2011, p. 34;Volk, 2013, pp. 42-43).
More recently, the need to have a more detailed and closer to real time understanding of the status of the electricity network, something particularly relevant to deal successfully with contingencies, has led TSOs to deploy wide-area measurement systems (WAMS). They consist of a network of devices that measure in real time quantities of interest on the transmission grid across a large geographic area. Particularly relevant among these devices are the phasor measurement units (PMU), which are able to detect relevant variables of the electrical wave flowing in the network at a high frequency (e.g., 30 times per second) and in synchronicity with other such units based on a global positioning system time signal (MIT, 2011, pp. 36-39;Volk, 2013, pp. 44-45).
At the transmission level, another set of monitoring technologies that is growing in importance is represented by those that enable dynamic line rating (DLR). In this case, sensors deployed over a power line measure a series of electric and environmental conditions like air temperature, solar radiation, wind speed and direction. With these data, it is possible to accurately estimate in real time the maximum amount of electric current that can effectively flow within the line without violating any security limit. Depending on the circumstances, such dynamic rating of line capacity can be up to 5-30 per cent higher than that estimated according to traditional approaches which relay on static assumptions and conservative criteria (MIT, 2011, pp. 45-6;Volk, 2013, p. 44;IRENA, 2020a).
Monitoring of transmission lines is more and more performed also with the help of unmanned aerial vehicles (drones), which can fly over the infrastructure and detect either possible damages after adverse weather events or the need to proceed with the pruning of local vegetation. This 'mobile' mode of data 6 This list does not aspire to be exhaustive. Indeed, one may argue that digital technologies perform additional functions as, for instance, the storing of data or their validation. For a discussion of digital technologies and the functions they perform from an economic point of view, refer to Goldfarb and Tucker (2019). Robert Schuman Centre for Advanced Studies Working Papers acquisition is particularly relevant as the electricity infrastructure can be located in hard to reach places or extend over a broad area (PwC andTractabel, 2019, ETIP SNET, 2018).
At the distribution level, the deployment of sensors for monitoring the physical infrastructure, as for instance those forming SCADA systems, is usually lagging behind. Nonetheless, it is important to observe that since the beginning of the 2000s, an increasing number of distribution system operators (DSOs) have undertaken the replacement of the electromechanical meters located at the premises of their customers with modern electronic meters that can more easily and precisely measure, store and communicate consumption and generation data as a function of time. Electronic meters can be of different types and perform different functionalities. Systems based on automated meter reading (AMR) allow utilities to read customer meters via short-range radio frequency signals that are usually captured from the street using specially equipped vehicles. Alternatively, systems based on advanced metering infrastructure (AMI) -typically referred to as 'smart meters'combine electronic meters with a twoway communication capability and are able to record and report consumption data at frequencies of one hour or less, directly to the entity in charge of performing the metering function, normally but not necessarily the DSO (MIT, 2011, pp. 132-137).

Technologies to transmit data
Different types of communication technologies and networks are employed to transmit data from data sourceslike sensors and metersto data sinkslike data and control centres, send back control signals or share data among different data sinks, both within and outside a specific organization. Depending on the specific requirements of the communication under consideration (e.g., data rate, latency, reliability and so on), alternative solutions have been introduced over the past decades. 7 Utility-owned networks are typically used for the transmission of operational measurement and control signals between control centres, substations and sensors deployed along electricity lines. These networks can be based on wired (e.g., copper or fibre optics), wireless (e.g., cellular, Wi-Fi, Zigbee, Bluetooth and so on) or radiofrequency and microwave communications. The power line itself can also be employed to send measurement and control signals, the so-called power line carrier (PLC). To send data between different data centres, dedicated commercial networks can also be used. Public communication networks, like the telephone network or the internet, are exploited too, in particular to transmit information such as price signals and generation schedules or to communicate with home energy networks. Satellite communications are employed in certain specific cases, for example to synchronise the data produced by PMUs. Finally, home and commercial premises networks, typically based on a local Wi-Fi or cable network provided by the customer herself, are used to connect end-use appliances and transmit control signals from the electric utility (MIT, 2011, pp. 199-204).
Among the various emerging communication technologies, the fifth generation of cellular networks, the so-called 5G, is claimed to be particularly relevant for the digital transformation of the electricity sector (Rhodes, 2020, p. 24). Due to its ability to support a huge number of connected devices and its low latency, such type of communication network looks particularly promising for the development of the internet of things (IoT), a system in which a myriad of devices generate and share data, and interact among them (IRENA, 2019a).

Technologies to analyse data
The deployment of cheap sensors and their connectivity ensured by the establishment of an increasing number of communication networks make available a growing amount of data of different types and 7 Different communication technologies and networks present different expected data rate/bandwidth, latency level, convenience of use, reliability and security level, and back-up power needs. They may also be more or less suitable depending on the distribution of the devices they need to connect: while some Wi-Fi networks are good to connect devices within a building, satellite communication networks can transmit signals over thousands of kilometres and connect dispersed devices.
Digitalization of the electricity infrastructure: a key enabler for the decarbonization and decentralization of the power sector European University Institute 5 formats. To effectively deal with it and to produce value from it, digital technologies aggregating and processing data are becoming more and more important. Energy companies and network operators are aware of that and have already adopted a broad set of software platforms, as for instance energy management systems (EMS) for TSOs, that combine the data coming from the various sensors deployed over the physical infrastructure or from external sources, evaluate the status of the infrastructure and support operational tasks.
Complementary to those more 'traditional' data analytics software and visualisation tools, significant expectations are assigned today to the algorithms that support artificial intelligence (AI) and one of its branches that is experiencing the fastest advancements, machine learning (ML). The basic idea, common to many other industries, is that AI can successfully manage big data produced in a complex system and use them to elaborate accurate predictions that guide better decision-making processes (Agrawal et al., 2018). Such idea is particularly attractive in the electricity sector due to its increasing decentralization and the need, but also the opportunity, to coordinate and optimize a large number of production, trade and consumption choices performed by a multitude of different actors who most of the time are individually very small. A number of pilot projects and early commercial applications are currently under implementation. They cover the entire value chain, from generators willing to predict their production potential or future wholesale price patterns, to network operators wishing to better forecast the status of their network in the coming minutes or hours. Retailers and innovative service providers are also investing in AI and ML with the purpose of predicting customers' consumption habits and developing targeted retention campaigns or demand response programmes (ENTSO-E, 2019b;IRENA, 2019b;Rhodes, 2020;. The abundant data provided by the sensors deployed on physical assets are increasingly used to create digital twins, that is virtual representations of those assets which can be employed to visualize them, monitor their operation, and support asset managers and maintenance crews. The increased visibility provided by digital twins can be then combined with AI and ML solutions to mimic the functioning of the physical assets, predict their future performance and effective need for maintenance, or test alternative operational decisions (ETIP SNET, 2018;PwC and Tractabel, 2019).
Another technology that can support decision-making and has recently received great attention is blockchain. Several start-ups, established electricity companies and network operators are investigating its application to support P2P energy trade, optimize the use of distributed energy resources (DERs), 8 manage electricity grids, finance new energy assets (e.g., RES-based power plants), issue and trade green certificates, manage charging of electric vehicles (EVs), or even perform disintermediated wholesale transactions (Livingston et al., 2018;Andoni et al., 2019;IRENA, 2019c). Permissioned blockchains look particularly promising in this regard because of their ability to deal with a larger number of transactions in a shorter time frame and with less energy consumption than permissionless blockchains. Nonetheless, this technology is not yet fully mature and implementation outside of pilot projects has been limited so far. Some even argue that, contrary to expectations, blockchain is not suitable to support transactive energy markets and its use may remain limited to a few applications (Hertz-Shargel and Livingston, 2020).

Technologies to implement control
An extensive number of smart switches, circuit breakers, relays, voltage regulators and capacitor banks have been installed and integrated within the management systems of network operators and network users or service providers, in order to control physical assets and implement remote commands, either 8 Distributed energy resources refer to 'demand and supply-side resources that can be deployed throughout an electric distribution system (as distinguished from the transmission system) to meet the energy and reliability needs of the customers served by that system. Distributed resources can be installed on either the customer side or the utility side of the meter' (Ackerman et al., 2001, p. 201). Consistently, DG, storage devices connected to distribution grids, electric vehicles and all the devices delivering demand response are DERs. Robert Schuman Centre for Advanced Studies Working Papers coming from a human decision or an entirely automated process. Endowed with microprocessors, such control devices are able to receive signals from control centres or from distributed sensors to automatically open a circuit or adjust the set points of a voltage regulator, thereby altering the functioning of the physical layer, without the need for any manual human action.
At transmission level, system integrity protection schemes (SIPS) and flexible alternating current transmission systems (FACTS) are among the most relevant technologies for network automation. The former are advanced and decentralized control systems able to take and implement decisions, as for instance the tripping of the circuit breakers at the ends of a power line. By reacting to local and widearea measurements, SIPS ensure a more intelligent and coordinated response to rapidly evolving system conditions (MIT, 2011, p. 47). FACTS, on the other hand, constitute a set of technologies employing power electronics that enable control of various system operating parameters, including volt-amperereactive support and power flow. Among FACTS devices, static volt-ampere reactive compensators (SVC) are increasingly deployed by transmission companies to better control voltage levels at the grid nodes and improve system stability characteristics (MIT, 2011, p. 48;Volk, 2013, p. 45). 9 At distribution level, grid automation is less developed, but progress has been significant in the past few years. DSOs have implemented distribution management systems (DMS) and outage management systems (OMS) that integrate control devices, enabling an automatic detection of outages and automatic attempts to solve or at least isolate the problem on the affected line, a function often called fault detection, isolation, and restoration (FDIR) or self-healing (MIT, 2011, p. 130-131). In order to improve voltage regulation and optimize power flows, DSOs are also deploying more sensors along electric lines that feed measured voltages back to the connected substations and activate control devices, adjusting the functioning of the transformers located in the substations or switching in or out of the circuit a set of capacitors. By implementing automatically this volt/volt-ampere reactive control function, voltage levels remain closer to the operating standard and power consumption can be reduced without affecting the power quality perceived by network users (MIT, 2011, pp. 131-132). Faced with the fast growth of DERs in many distribution networks, some DSOs are also deploying smart control devices to be able to dispatch those resources and possibly disconnect them if they cause congestions or represent a threat to the functioning of the distribution network. In this case, control devices are usually integrated in distributed energy (resource) management systems (DEMS or DERMS), a new and still evolving system used for active management of DERs from a single unified platform (BNEF, 2017, p. 48).
We can conclude this overview by mentioning that digital control devices are more and more present in many of the assets and appliances that energy companies, service providers and final customers connect to the electricity grid, both in front of and behind the meter. In this context, home energy management systems (HEMS) able to integrate those control devices and ensure an automated monitoring and control of the appliances within the premises of a final customer represent an innovative technology with a very positive outlook (BNEF, 2017, p. 2;PwC and Tractabel, 2019, pp. 63-83).

Overview of the data layer
The application of the digital technologies mentioned in Section 2.1 is leading to the generation and use of a growing amount of data in the electricity sector. However, such data layer is far from homogeneous and its organization and exploitation are often quite fragmented. The availability and quality of data on some parts of the sector are higher than on others. Accessibility to data often remains an issue. While in some cases, access to data and their use by third parties is eased thanks to higher levels of standardization regarding interfaces, data models and formats as well as an enabling policy and regulatory framework, in other cases accessibility and interoperability remain limited (Rhodes, 2020;Morris et al., 2020). This situation is the result of the differentiated rate at which digital technologies have been applied to the 9 The development of offshore wind generation and the construction of high voltage direct current lines, both onshore and offshore, further increase the importance of power electronics and control devices at the transmission level. 7 various segments of the electricity sector. It is also a result of the nature of the electricity system as a 'system of systems', where a multitude of smaller and larger systems that generate, process and share data have gradually emerged over time, not necessarily in a coordinated way, and are frequently subject to different legal and regulatory frameworks (Schittekatte et al., 2020a, p. 122).
Data in the electricity sector can be broadly classified in four categories: network data, market data, consumer data and external data (see Table 1). 10 Each of these categories refers to a different domain and is usually characterized by a different level of development and organization, including the relevant regulatory framework and governance. Network data, sometimes also referred to as system data, include structural data on the localisation and technical characteristics of the assets that form the electricity network, both transmission and distribution, and the larger generation or consumption units. They also include the data on the operating conditions of the system, for instance the power flowing in a node of the network or the injections by a large power plant within an hour. Operational data can be both actual and forecasted. Network data are necessary for the operation and planning of the electricity system and represent an important input also for the functioning of electricity markets. Network data are usually generated or collected by system operators and made available to other (interconnected) system operators or to market players to increase coordination in the electricity system (see Section 5.1 and 5.2). Due to their importance in system and market operation, network data often benefit from a higher level of harmonization and accessibility imposed by regulation on the entities that generate and collect them, with the possible and partial exception of distribution network data. The ultimate result is a more effective data exchange and data exploitation. 11 Market data include data on bids and offers by market participants for electricity and related products at the wholesale level (e.g., financial transmission rights). They also include data on the prices agreed upon and the quantities exchanged, both within and outside organized markets, as well as data on imbalances and settlement. Market data are important to ensure transparency and enable market 10 Energy data can be classified in different ways, depending on the purpose of the classification and the criteria adopted.
Nevertheless, the four categories mentioned in the text represent a fairly common way to cluster them. 11 Security considerations and commercial sensitivity sometimes can justify a restricted access to network data. Robert Schuman Centre for Advanced Studies Working Papers monitoring. They are also an essential input for many of the analyses and planning conducted by system operators, both on the long and short term. Planning long-term grid expansion, for instance, can be justified by market data that consistently reveal the existence of unsatisfied demand for more transfer capacity in a part of the network. Market data are usually generated or collected by market operators, e.g., power exchanges, and made available to market participants, relevant system operators, and possibly other market operators (see Section 5.2). These data are usually also made available to designated bodies, either public or private, that are tasked with market monitoring. Similar to the case of network data, market data have received significant attention in the past and are usually characterized by a high level of access and harmonization that support their exchange and exploitation. 12 Consumer data include metering and consumption data by final consumers, either cumulative or by time intervals. They also include data on the output of distributed generation units connected behind the meter or the usage of appliances involved in demand response programmes. Data on billing and payments, as well as data on the customer (e.g., identity of consumers) and metadata (e.g., type of business activity, address and so on) are included among consumer data. This data category has witnessed a quantitative and qualitative explosion in the past few years due to the deployment of smart meters. While in the past, consumption data were available only at a monthly interval or even less frequently, it is possible today to have data on the electricity consumed by a final consumer every thirty minutes or less.
While consumer data were traditionally of limited interest and were mostly used by the DSO and the energy supplier for billing purposes, in recent years the increasing granularity of consumer data, the possibility to extract more valuable information from them and the new rights attributed to final consumers have increased the attention by market participants, in particular those willing to provide new services or innovative pricing schemes (see Section 6.1). However, fragmentation is a typical feature of this part of the data layer in the energy sector and data exchange is not always seamless. Although data ownership is usually attributed to the final consumer, consumer data are actually collected by DSOs and/or energy suppliers, and are subject to different data management models (DMMs) at the national level, which reflect the fragmentation of the electricity distribution system and the different practices developed over time (Schittekatte et al., 2020a, p. 138). In particular, the reduced level of harmonization and data accessibility is often related to different choices made in the attempt to balance distinct legitimate but sometimes conflicting interests and public goals, as for instance the interest of consumers to data privacy or the interest of commercial parties to access consumer data on a level playing field with incumbents. The consequence of this fragmentation is the existence of barriers to new entrants and the difficulty, at least for the time being, to fully exploit consumer data to spur innovation in retail markets.
Finally, external data include typically weather and climate data, as well as socio-economic data, as for instance data on gross domestic product or population growth. These data are collected by a variety of actors, often outside the electricity sector, and are used in different ways to forecast energy production and consumption patterns, both on the short and long term. External data are quite heterogeneous and present different levels of harmonization and accessibility. In some cases, access to external data is open, for instance in the case of public statistics, while in other it is restricted by private agreements between the data provider and the data user.

Impact on infrastructure planning and maintenance
The vast availability of data that are produced nowadays and the possibility to transfer and rapidly analyse them have important implications on the way the electricity infrastructure, both power plants and networks, are planned, built, operated and maintained. The deployment of sensors, 12 Commercial sensitivity of certain market data justifies restricted access in some cases. Still, data are frequently accessible when aggregated.
Digitalization of the electricity infrastructure: a key enabler for the decarbonization and decentralization of the power sector European University Institute 9 telecommunication networks, data centres, online platforms and algorithms for AI at decreasing costs has the potential to reduce the expenses associated with those activities, while improving the performances of the related physical assets. The International Energy Agency (IEA) estimated that the enhanced deployment of existing digital technologies to all power plants and networks at the global level would result in a yearly reduction in total power systems costs of about USD 80 billion (IEA, 2017, pp. 78-81). Of this amount, close to 20 billion would be saved on average each year in operation and maintenance (O&M)to have a reference in mind, USD 300 billion were roughly spent for O&M activities in 2016. The reduced wear and tear of the physical assets, both generation and networks, would allow an extension of their lifetime, deferring the need for new investments and saving an additional 54 billion per year. The improved operation of the assets was estimated to increase the electricity output per unit of fuel and reduce total network losses by 5 per cent, providing additional savings of USD 6 billion. Furthermore, the deployment of digital technologies could reduce the frequency and duration of unplanned outages and downtime, but no estimate about the resulting cost savings was provided. Numbers in the same order of magnitude were equally calculated by the World Economic Forum and Accenture (2016). Based on a survey of industry experts and by extrapolating data from a sample of energy utilities operating in OECD countries, they estimated that the potential value generated for the industry and the society at large by digital technologies in improving asset performance management, field workers' productivity and asset planning would be almost USD 1000 billion over the period 2016-25, with asset management offering the greatest opportunities to the industry (WEF and Accenture, 2016, pp. 10-12) In this section, we illustrate the impact of digitalization on infrastructure planning and maintenance. Section 3.1 describes how digital technologies can provide a better understanding of the future use of a physical asset and plan it in a way that such use can be maximized. Section 3.2 focuses on how digital technologies can improve the forecasting of the maintenance needs of physical assets and support maintenance activities.

Infrastructure planning
Digitalization allows more informed decisions about the planning of new infrastructures for the generation, transmission and distribution of electricity. Power plants and electricity grids are long-lived physical assets, whose use and the revenues they generate depend on several factors. Among these are the spatial and temporal evolution of electricity demand, the availability and price of primary energy sources, including renewable ones, or the presence of complementary or substitute infrastructure. Considering all these factors is important to site and size an investment properly. However, developing adequate scenarios and cost-benefit analyses in the electricity sector has become increasingly difficult in recent years because of the vertical disintegration of the industry, the growing uncertainty over the evolution of energy demand and the increasing variability of energy production, especially that dependent on weather conditions (Fox-Penner, 2020). Digital technologies can help to address such uncertainty by enabling the collection and analysis of (big) data and the simulation of the future use of an existing or new infrastructure under different conditions. The application of advanced weather forecast techniques that rely on the collection and analysis of data is particularly important in guiding the decision on where and how to place a wind or PV power plant, since the availability of the primary energy resource is typically one of the main uncertainties for this type of investment (IRENA, 2020b). By predicting the (typical) weather conditions in a specific site over the long term and combining them with the planned characteristics of the power plant, these techniques enable better estimations of the amount and temporal distribution of the electricity output.
The use of sophisticated simulation tools is important also for network companies and system operators that are mandated to ensure the adequacy and security of the electricity system in the short and long term. When developing their expansion plans, network companies need to justify their investment decisions, whose cost will be borne by ratepayers, in terms of additional transfer capacity Robert Schuman Centre for Advanced Studies Working Papers offered to the market, increased system reliability or integration of renewables. Making such assessment in an effective and transparent way without the use of simulation tools that model the physical grid under different scenarios and dynamically take into account the impact of weather conditions on generation patterns and market prices would be impossible (Fox-Penner, 2020). A good example in this regard is provided by the Ten-Year Network Development Plan (TYNDP) that European TSOs are mandated to produce every two years. ENTSO-E, the association that gather electricity TSOs in Europe, leads the process, recently also in collaboration with the twin association for gas TSOs, and develops a set of long-term scenarios of the European energy system. Then, it performs a calculation of the benefits of a proposed investment in new transmission capacity by running a chronological stochastic market simulation of all hours of a target year. A pan-European climate database is used to ensure that weather conditions are considered in the simulation. 13 Digitalization affects the planning of electricity grids also by allowing grid operators to consider new options in alternative to the traditional reinforcement of power lines and upgrading of transformers. Instead of investing in more 'iron and copper', grid operators can include in their grid development plans the use of so-called 'non-wire alternatives', that is the procurement of ancillary services and flexibility by grid users, which may efficiently solve temporary or rarely occurring congestions and voltage problems (Volk, 2013;Nouicer and Meeus, 2019, pp. 46-58;Baker, 2020).

Infrastructure maintenance
Digital technologies allow an improvement in O&M activities and a reduction of their costs. Central to this possibility is the development of so-called digital twins and solutions of augmented reality, the use of drones, and the deployment of self-healing devices on the physical infrastructure.
First, the development of digital twins enables predictive maintenance. The application of sensors to physical assets like power plants or power lines and their connection to the cloud allow to develop a digital twin of these physical assets and the identification of their effective maintenance needs. By means of AI and machine learning techniques, the digital twin is tested and required interventions on the physical asset are predicted based on its actual state as opposed to conservative assumptions and maintenance schedules (PwC and Tractabel, 2019). The role of predictive maintenance is particularly relevant when the asset is located in a remote or hard to reach position, as for instance an offshore wind turbine or a transmission power line crossing difficult terrain. In this case, implementing on-site maintenance activities only when strictly necessary could save significant costs.
Second, solutions of augmented reality improve the visualization of a physical asset by both on-site workers and remote support teams. By ensuring a seamless sharing of information, augmented reality facilitates the tasks assigned to maintenance and repairing crews, which can be leaner in terms of staff and act faster (ETIP SNET, 2018).
Third, monitoring of the asset conditions can be performed with the support of unmanned aerial vehicles as well. These devices can provide accurate images of the conditions of the asset and the surrounding environment. Various European grid companies have already started experimentations with drones to inspect electric lines with positive results so far; in some cases, they are also using them to implement maintenance activities (PwC and Tractabel, 2019, pp. 177-182; ETIP SNET, 2018).
Finally, digital technologies do not only enable preventive maintenance but also help when a fault occurs. In the case of grids, for instance, the availability of granular data on the conditions of the asset speeds up the detection of the point of failure and may allow the implementation of automated selfhealing procedures, whereby the grid automatically solves the problem, minimizing in this way the duration of service interruption. 13 More information on the TYNDP and its methodology can be found on the ENTSO-E website. Useful documents are

ENTSO-E (2019c) and ENTSO-E (2020a).
Digitalization of the electricity infrastructure: a key enabler for the decarbonization and decentralization of the power sector

European University Institute 11
Statistics about the level of cost reductions and other benefits of improved O&M are not usually available. However, some industry sources, for example, estimate that the application of digital technologies to wind farms can reduce the cost of O&M activities by 10 per cent, increase electricity production by 8 per cent and decrease curtailment cuts by 25 per cent (BNEF, 2017, p. 18).

Impact on the operation of the infrastructure
By enhancing the observability, predictability and controllability of the electricity infrastructure, digitalization enables a more efficient operation of the existing capacity without causing additional risks for the reliable and secure functioning of the system. This is true for any asset of the supply chain, from generation to consumption. However, the case of electricity networks is particularly interesting because their operation is constrained by the need to constantly balance power injections and withdrawals while respecting security limits. The importance of these constraints and the externalities that the use of the network by any generation or load unit determines explain why the operation of the electricity network is normally assigned to a single actor by virtue of a legal mandate. It also explains why such system operator, charged with the continuous and secure functioning of the system, is used to operate the network in a prudent way. Given the fact that even a small failure in a power plant or on a power line can trigger significant consequences and given the fact that interaction with network users suffers from a series of limitations, installed network capacity is never entirely used to transmit and distribute electricity under normal conditions ('N-1' security criterion).
Digitalisation changes all of this and allows transmission and distribution system operators to manage more efficiently the supply and demand of network capacity. By doing that, system operators can ensure a higher utilisation of the infrastructure and spread the fixed costs they incur over a larger usage. Remarkably, the transition towards a more decentralized and decarbonized electricity system and the electrification of certain end uses like heating and transport increase the importance of such evolution in the operation of electricity networks (Volk, 2013). Certain renewable-based generation assets or certain appliances like EV charging units present in fact a highly volatile and sometimes peak-coincident use of network capacity. In addition to that, their rapid deployment is not always consistent with the existing network and its ability to satisfy their capacity demand at any time. As a result, system operators frequently face an increasing demand for (new) network capacity, while not necessarily observing any growth in the total amount of energy passing through the grid. Together, these trends can lead to escalating costs and to new risks if the old approaches to system operation and network planning are maintained. 14 In this section, we consider how the application of digital technologies supports a change in the operation of the electricity infrastructure and ultimately enables a more efficient use of existing capacity. Section 4.1 describes how digitalization allows system operators to offer more of the existing capacity without causing congestions and more risks to system security. Section 4.2 shows how digitalization permits to steer demand for network capacity away from peak to off-peak times.

Managing capacity supply
In a liberalized electricity system, the system operator manages the supply of network capacity by performing a series of functions. First, it calculates the maximum physical network capacity available to transport electricity along the various parts of the network, without breaching any security limit. Second, it allocates such available network capacity to the different network users and ensures that their 14 An expansion of network capacity that satisfies capacity demand under any condition can be socially inefficient because part of that additional capacity would remain idle most of the time and alternative solutions to deal with peaks might be less expensive. Moreover, such expansion could also pose financial risks for the network operator since a change in demand patterns or in the regulatory framework may seriously undermine its cost recovery. expected use of the network and the resulting power flows are compatible with the physics of the network itself. 15 Finally, it intervenes when for any reason an imbalance between generation and consumption in the system occurs or a congestion in an element of the network emerges. In such case, the system operator implements a series of actions that aim to adjust supply and demand of network capacity to the changing conditions, while maintaining a secure functioning of the system. To preserve operational security, the system operator can alter the topology of the network, redispatch some of the generation units, or, as a measure of last resort, can disconnect certain generation or load units from the rest of the system. 16 In the past, system operators were used to manage capacity supply in a rather prudent way. Since they had only limited information on the effective conditions of the system and its likely evolution in the following minutes, hours or days, and since they could only implement corrective measures in a rather slow and imprecise way, a conservative approach was typically applied to the calculation of the available network capacity. Electric lines were rated in a static way, considering the worst possible external conditions, and hefty margins were added to be sure that unanticipated power flows and contingencies could be successfully dealt with. As a result, although unused network capacity was effectively there, generators and load units were in some cases prevented from using the network, or even not allowed to connect to it.
Digital technologies permit nowadays to have more abundant and closer to real time data relative to the network, the surrounding environment and the electricity markets, allowing system operators, especially at the transmission level, to understand better and in a more granular way what is the status of the system. By implementing dynamic line rating (DLR) solutions, system operator can for instance better assess what is the effective maximum amount of power that can safely flow through a line in a given moment (IRENA, 2020a). By developing sophisticated grid models, system operators can analyse system dynamics, simulate expected power flows and identify possible constraints and criticalities . By applying AI and machine learning, they can elaborate more accurate and rapid predictions on the short-term evolution of system conditions, which take into account possible interdependencies in the calculation and allocation of network capacity, and dimension reserves for operational security on the basis of the specific needs of the system in a given time frame (de Vos at al., 2019; IRENA, 2019b). Digitalization positively affects system operation also by enhancing the controllability of the physical layer and allowing system operators to implement remedial actions in a faster and more granular way. While in the past interventions on the network often required human action, modern technologies frequently enable the automated and rapid implementation of the needed solutions. Remarkably, they increasingly provide the possibility to target not only the largest assets, but the small ones as well. By opening or closing a circuit or by adjusting the voltage output of a transformer, system operators can modify the topology of the grid and adjust the network capacity offered to network users within a very short notice. They can also send instantaneous re-dispatching orders to some generation units, asking them to increase or decrease their generation output, thereby altering the flows of electricity and reducing congestions on some branches of the network. Finally, they can also remotely disconnect certain network users on the generation or consumption side, which is equivalent to curtail the supply of network capacity to them.
Today, all these solutions are common practice at transmission level, but they are gaining importance within distribution networks as well. With the support of ICT, DSOs are gradually moving from a rather 15 This function is usually performed in coordination with the operator(s) of the electricity market(s). In some cases, the system operator is also the market operator, while in other cases the two entities are separated. See Section 5.1 for more on this. 16 Redispatch means any measure activated by one or several system operators by altering, in a predefined way, the generation and/or load pattern in order to change power flows in the system and relieve a physical congestion (Schittekatte et al., 2020a, p. 41).
Digitalization of the electricity infrastructure: a key enabler for the decarbonization and decentralization of the power sector European University Institute 13 passive management of their network to a more active one, in which they resemble more and more TSOs. 17 The optimistic scenario just described must be nonetheless somewhat toned down. If digital technologies provide indisputably advantages to system operators and increase their ability to observe, predict and control the network, it must be noted that the integration of different electricity markets at the regional level, the penetration of variable renewables, the deployment of distributed generation and the growing presence of new loads and distributed storage units are making the life of system operators harder. The net effect is unclear and some argue that even with the new digital technologies system operators must change the way in which they pursue security of supply if they want to fulfil their mandate in an effective and efficient way (Bialek, 2019).

Managing capacity demand
In a liberalised electricity system, the system operator manages the demand of network capacity mainly by pricing the access of network users to the network. 18,19 Several options are available in this regard. Access can be granted to a network user up to the maximum capacity of the individual connection to the public grid and the pricing based on a tariff defined ex ante. Access can also be granted conditional upon the results of the electricity market. In the latter case, the cost to access the network is defined simultaneously with the price of the electricity that the network user is going to inject or withdraw from the network. In both cases, the pricing of capacity can present different degrees of granularity and reflects to a different extent the effective costs for the system to supply such capacity. In particular, the price to access the network can be fixed over time and space or be variable, depending on the time of the day or the precise place in which capacity is demanded. It can represent some form of 'average cost' of supplying network capacity to all network users or be strongly related to the actual costs caused by the choices of an individual network user. Depending on the characteristics of the pricing rules adopted, different economic incentives are provided to the users of the electricity network.
As in the case of capacity supply, before the introduction of digital technologies system operators had difficulty in managing capacity demand in any sophisticated way. Data on the use of the network were often lumpy, especially when referring to smaller users. While for a large power plant or industrial site, an accurate profile of the power exchange with the network was eventually available, in the case of a household the system operator was used to know only the cumulative energy consumption over a few months and/or its absolute peak demand over the same period. Moreover, even if more data were available and more sophisticated prices could be computed, the difficulty to timely convey those prices to the network user and the limited possibility for her to efficiently react and adjust her choices on how to use the network would limit the effective possibility to steer demand for capacity and improve the load factor of the network. As a result, system operators were used to apply rather 'crude' pricing rules, Robert Schuman Centre for Advanced Studies Working Papers whose goal was mostly to ensure the recovery of network costs rather than the provision of efficient and fine-tuned signals to the users of the electricity system. 20 Digitalization provides the technical capability for system operators to better manage capacity demand. First, the gradual deployment of smart meters allows the system operator to be aware, often close to real time, of the power injected and withdrawn from the grid by network users with a high level of time and space granularity. Second, advanced software and increasing computational power enable the calculation of large numbers of different prices that reflect specific supply and demand conditions and possibly take into account network constraints. Third, bi-directional communication networks, automated energy management systems and smart appliances permit the transmission of more sophisticated price signals to network users and allow them to efficiently react and adjust their demand for network capacity.
At transmission level, the application of digital technologies is already steering network demand. Large power plants or load centres directly connected to the transmission grid face a price for the electricity they sell to or buy from the grid that considers, to various extents, the balance between supply and demand of electricity on the market and the possible constraints in the electricity network. In this regard, the most advanced solution is locational marginal pricing (LMP), which is used in some of the North American wholesale power markets (Volk, 2013). In these markets, the system operator, which is also the market operator, collects all the bids and offers for electricity by the network users and computes every few minutes thousands of prices, one for each node of the transmission network. These prices reflect not only the demand and supply for electricity but also the available transmission capacity connecting each node and the energy losses that each injection or withdrawal of power generates at the margin. Faced with these prices, network users have strong incentives to consider the congestion they may cause when accessing the network and the value for the system to produce or consume electricity at that moment in that location. This encourages them to adapt, in the short term, their production or consumption patterns and to relocate, in the longer term, their generation or consumption assets to a part of the network with more unused transmission capacity.
Outside North America, most electricity markets, including those in Europe, usually do not apply LMP but follow a zonal pricing approach, where prices are not computed node by node but per 'zone'. Physical transmission capacity within one zone is treated as infinite (the so-called 'copper plate') and all network users within the same zone 'see' the same price for electricity. If properly designed, zones are aligned with the physical reality of the grid and the configuration of their borders reflect structural congestions. In this case, the zonal approach enables capacity demand management for cross-zonal lines, that is for the lines that connect two zones. However, zones are not always properly designedin Europe, for instance, they often simply reflect national bordersin which case the zonal approach 'hides' intra-zonal congestion and requires ex-post interventions by the system operator to manage congestion within zones .
At distribution level, management of capacity demand is normally less developed and most users, both on the demand and supply side, are still passive. They are typically neither aware of the real-time status of the system nor do they face price signals reflecting this status. Research on distribution locational marginal pricing (DLMP), that is LMP applied to distribution network users, is ongoing (Caramanis et al., 2016). Current practice shows, however, that most generators and loads connected to distribution networks are allowed to access the grid up to their maximum connection capacity, irrespective of the conditions of the grid, and pay a network charge that is based on average costs (Burger et al., 2019a;Burger et al., 2019b;CEER, 2020a;CEER, 2020b). This charge is typically the same for every user connected to the same voltage level, with the same connection capacity and part of the same user class (e.g., residential users, small businesses and so on). Only in extreme cases, the DSO intervenes 20 The adoption of 'crude' and not fully cost-reflective pricing rules can be explained also in terms of the historical perception of electricity supply as a 'public service', for which any discrimination among users must be avoided.
Digitalization of the electricity infrastructure: a key enabler for the decarbonization and decentralization of the power sector European University Institute 15 and directly manages capacity demand, for instance, by curtailing a DG unit or shedding the load of a neighbourhood.
Even if the application of DLMP is not in sight, digitalization is promoting the development of demand response (DR) and the 'activation' of smaller network users, with significant benefits in terms of reduced peak demand and deferred investments in generation and network capacity (MIT 2011;Cook, 2011;IEA, 2016;IEA, 2017). Demand response takes two forms. First, price-based or implicit DR means that the deployment of smart meters enables every network user, including households and small businesses, to be exposed to time-varying tariffs for electricity and benefit from autonomous decisions to shift demand off-peak. Second, incentive-based or explicit DR means that the same smart meters, possibly together with other monitoring and control devices, enable the introduction of a broad series of programmes and products where network users receive economic incentives other than the normal tariff for electricity to alter their consumption profile at times of high wholesale prices or when system reliability is jeopardised. Both forms of demand response are not new for large energy consumersinterruptible contracts for industrial users were common already in the 1980sbut advances in ICT and reduction in the cost of sensors and control devices have made possible their further application to commercial and residential network users. Time of Use (ToU) and dynamic electricity rates are now common practice in Europe and various parts of the US; similarly, explicit DR is well established in North America and is experiencing growth in some EU Member States as well (Hurley et al., 2013;SEDC, 2017;Sioshansi, 2020b). 21

Supporting coordination in a fragmented system
The secure and continuous supply of electricity requires the constant coordination of all the injections and withdrawals of power into and from the electricity grid. Any imbalance between the overall supply and demand of power leads to a departure of system frequency from the reference value -50 Hz in Europe and 60 Hz in the USthat can quickly turn into a blackout if a narrow band is exceeded. Similarly, injections and withdrawals of power that are not compatible with the physical capacity of the network can lead to the overloading of one or more lines and the rapid opening of their circuit breakers, culminating in the disruption of power flows and the possible occurrence of dangerous disturbances in a broad geographical area. 22 Before the restructuring of the electricity industry in many countries around the world during the 1990s and 2000s, the necessary coordination between generation and consumption was performed internally by the vertically integrated electric utilities. By simultaneously controlling (almost) all the generation assets and the entire network over a given territory, utilities were able to plan and implement in a coordinated way the various operational decisions and ensuremost of the timethe secure and continuous supply of electricity. At that time coordination was achieved via hierarchies and the use of relatively simple communication tools, transmitting to the various power plants the orders of the utility's central dispatch algorithm. By opening the sector to new entrants and by unbundling network activities from generation and supply, the liberalization of the industry called for new mechanisms to coordinate a growing number of generators, consumers, intermediaries and network companies. Markets and prices 21 Even when equipped with digital tools, the management of capacity demand is not necessarily an easy task. In particular, explicit demand response presents difficulties whose solution is not so obvious. One of these difficulties is the definition of the baseline against which demand response is measured (Rossetto, 2018). 22 Coordination is equally important on the longer term: the decisions on how much and what type of generation, storage and consumption assets to deploy and on where to locate them must be somehow coordinated if one wants to ensure system adequacy and reliability, not to mention cost-effectiveness. The preparation of grid development plans and the publication of medium to long-term adequacy reports by network companies and system operators are the typical forms in which longterm coordination is pursued in liberalized electricity systems, normally under the oversight of regulatory authorities and open to the consultation of all stakeholders. However, they are not considered in this chapter due to limited space and the less transformative impact of digitalization. Robert Schuman Centre for Advanced Studies Working Papers had to replace hierarchies and direct orders. More recently, the integration of markets at the supranational level, the penetration of variable renewable energy sources (vRES) in the generation mix, the deployment of DERs, the activation of smaller energy consumers 23 and the blurring of the boundaries between the different energy sectors and vectors pose new challenges and call for the establishment of additional coordination mechanisms that were not anticipated during the liberalization era (Burger et al., 2019b; see Section 7 for more on this).
By reducing the cost of gathering data and information from different sources, and processing, storing and transmitting them to different users, digital technologies support the introduction and functioning of a broad set of coordination mechanisms. Central to this is the concept of platform, a 'digital space where users can communicate and interact with each other and get […] access to products, services, or more broadly 'resources' provided by peers or organisations' (Kloppenburg and Boekelo, 2019, p. 68). 24 In the electricity sector digital platforms are key to share data and information among the numerous actors that operate along the supply chain and coordinate their autonomous decisions. These platforms are of different types. Without any ambition to be exhaustive, we could say that some platforms provide the marketplace where sellers and buyers of electricity or ancillary services meet and strike deals; other platforms facilitate the functioning of such marketplaces by enabling a simplified access to relevant information on the electricity system and wholesale transactions to all the interested parties; while another type of platforms supports the management of consumer data.
In this section, we provide an overview of how digitalization helps the coordination of the various actors involved in a liberalized electricity system via the establishment of digital platforms. Section 5.1 describes the use of platforms to coordinate the decentralized supply and demand of electricity at the wholesale level. Section 5.2 presents platforms that aim to increase the transparency and integrity of wholesale markets. Section 5.3 finally takes account of the growing importance of platforms to manage information related to retail markets and final consumers.

Platforms to coordinate supply and demand of electricity at wholesale level
Digital platforms played an important role in the liberalization of the electricity sector and it is no coincidence that the establishment of the first open markets for electricity occurred only a few years after the ICT revolution of the 1970s and 1980sthe UK Power Pool started operation in 1990, while Nord Pool enabled trade of electricity in Norway and Sweden from 1996 onwards.
As we have already mentioned above, electricity is a very special commodity, whose physical delivery requires the use of a common infrastructure and whose value depends on three 'elements', that is time, location and flexibility (Schittekatte et al., 2020a, p. 5). Moreover, due to its specific characteristics electricity is mainly traded ahead of real time with only balancing occurring on the spot (Schittekatte et al., 2020a, p. 6). To coordinate the decentralized supply and demand of electricity by a plurality of actors and ensure the consistency of such exchanges with the continuous and secure functioning of the system, several platformsin fact new marketplaceswere established in the 1990s and 2000s. These platforms typically took the form of power pools or power exchanges, open only to large and professional parties and focused on day-ahead transactions. Although many details differ from case to case, the users of those platformsbe they utility-scale generators, large consumers or tradersare able to make their offers and bids for energy for every single hour or half hour of the following day. 25 23 An active customer or consumer refers to a final energy consumer that is engaged in the consumption and production of electricity. In this sense active customer is a synonym for prosumer. EU legislation formally recognizes this entity and explicitly mentions other activities in which an active customer can be involved: she can store self-generated electricity, sell it and participate in flexibility or energy efficiency schemes (Nouicer and Meeus, 2019, pp. 70-72). 24 In this chapter, platforms do not necessarily refer to multi-sided markets. 25 In a power pool all producers and buyers of electricity are obliged to submit all their bids and offers to the organized market, while in a power exchange bilateral transactions outside the organized market are possible as well. In the latter case, the Then, the operator of the platform, the so-called market operator, defines prices and quantities of electricity for each hour or half hour of the following day and, in close coordination with the system operator, allocates the right to inject or withdraw electricity from the network to the various market participants, while taking into account system's constraints. 26 Over time, these marketplaces have become more sophisticated and have expanded to cover different time frames. On the one hand, forward markets allow today the trading of energy and transmission capacity months or even years ahead of delivery, while, on the other hand, intraday markets enable market participants to adjust their position after the results of the day-ahead market have been communicated and before the final gate closure, usually only a few hours ahead of actual delivery. Markets for balancing and ancillary services permit system operators to procure in a potentially competitive and efficient way the flexibility necessary to ultimately preserve, in real time, the balance between supply and demand and solve possible local congestions. 27 These developments are particularly important given the increasing amount of vRES connected to the system. Because of the difficulty to predict and, even more important, to dispatch those sources according to a schedule defined several hours before delivery, it is essential that market participants are able to adjust their commitments at a later stage and that the system operator is able to procure in a cost-effective way the reserves and balancing energy needed to cope with such variable generation (Sioshansi, 2020b).
New platforms and pricing algorithms have played an important role also in the gradual integration of wholesale markets at the regional level. In order to enable trade between interconnected systems run by different system and market operators, several coordination mechanisms are necessary (Baritaud and Volk, 2014;Rossetto, 2017). Among them, there are the mechanisms and rules for calculating and allocating the available cross-border transmission capacity and those that compute the prices and quantities of electricity that are exchanged in the various hours and locations. As an example, in the EU the integration of markets at the regional level has been pursued by agreeing on a set of detailed rules, the so-called network codes, and by establishing a series of platforms (Meeus, 2020). A single pan-European platform, run by the Joint Allocation Office (JAO), which is a service company owned by 25 TSOs from 22 countries, is responsible for the allocation of cross-zonal transmission capacity via regular auctions. In the day-ahead time frame, a common price algorithm, called EUPHEMIA, is used to couple most of the electricity markets in the EU and simultaneously calculate prices for electricity and implicitly assign cross-zonal transmission capacity. This is a major achievement that only developments in ICT made possible, since the algorithm must take into account for each hour of the following day the multitude of bids and offers made by market participants dispersed from Portugal to Finland, the transmission capacity available between each market zone and, on top of that, the fact that slightly different rules may apply in the various national markets (for instance the fact that in Italy demand faces a single price while supply faces zonal prices). In a similar fashion, the Cross-Border Intraday Market Project (XBID) provided the basis for the development of a common IT system used to gradually couple the intraday markets at the continental level and enable the continuous matching of bids and offers parties involved in bilateral transactions are obliged only to communicate the profile of injections and withdrawals they have agreed on but not the price of the deal. For an extensive presentation of the fundamental economics of electricity markets, see Stoft (2002). A more recent overview that illustrates the solutions developed in a series of countries is Glachant et al. (2021). 26 The market operator and the system operator are two functions that can be performed by the same actor or by different actors. When performed by different actors, rules are normally in place to ensure that they coordinate their activities, especially when the market in question is not purely financial but has a physical dimension, like the day-ahead or the intraday ones. 27 Due to the importance of balancing and ancillary services for the security of the system and due to the necessity of perfect coordination under short time frames, balancing energy and the various ancillary services were traditionally procured by the system operator via a series of administrative mechanisms, imposing, for example, technical obligations on certain generation units or by signing long-term procurement contracts. The development of digital technologies addresses these concerns and offers today the possibility to replace traditional administrative approaches with market-based solutions that can foster competition among the providers of ancillary services and reduce the cost of system balancing. Robert Schuman Centre for Advanced Studies Working Papers entered by market participants located in different countries only a few hours ahead of real time (Schittekatte et al., 2020a, pp. 58-59). Finally, four common platforms (TERRE, MARI, PICASSO and IGCC) originated by the voluntary cooperation among various European TSOs were later chosen to serve the exchange of balancing energy from different types of reserves and support the imbalance netting process at the supranational level. Common IT systems and interoperability play a crucial role in the well-functioning of these platforms (ENTSO-E, 2020b).

Platforms to increase market transparency and trust
Digital platforms can support the decentralized coordination of the actors participating in electricity markets by collecting, storing and making available to them a large amount of network and market data (see Section 2.2 for the classification of data in the electricity industry). By increasing market transparency, this type of platforms reduces the risks and uncertainties associated with the determinants of electricity prices (e.g., planned and unplanned generation outages, demand forecasts, transmission capacity available and so on) and promote the development of efficient and liquid markets. These platforms could be particularly helpful for new entrants since they remove information asymmetry and level the playing field between newcomers and incumbents. Competent authorities can leverage the data provided by these platforms and improve market monitoring and the detection of anti-competitive or abusive behaviours by market participants. In turn, this can increase trust in the well and fair functioning of the market and attract further players and investments. An improvement in effective competition and a reduction of prices for final consumers would be the ultimate and positive result for society (De Francisci, 2014). 28 In the EU, ENTSO-E was mandated to create one of such platforms by Commission Regulation (EU) No. 543/2013 (also 'Transparency Regulation'). The ENTSO-E Transparency Platform serves for the central collection and publication of 'close to real time' data on load, generation, transmission, balancing, outages, congestion management and system operations. The purpose of the platform is to provide market participants, other stakeholders and the interested public with timely, free and openly accessible information on the state of the electricity system. ENTSO-E is the platform operator, while primary data owners (PDO) like TSOs, DSOs, generation and consumption units, operators of direct current links and power exchanges (PX) provide the data to the platform, often not directly but through intermediaries, called data providers (DP). 29 ENTSO-E has also developed a distributed software platform (ECCo SP) to collect and distribute the data in a seamless way. ECCo SP is an exchange service bus that represents the foundation for exchanging data across business applications in the power system and facilitates the secure communication of a wide variety of data. All European TSOs use ECCo SP to send data to the Transparency Platform, making use of its automated processes and functionalities (Schittekatte et al., 2020a, pp. 126-127).
Another example of platforms that facilitate the well-functioning of wholesale markets are the Inside Information Platforms (IIP), developed in the EU to address the transparency requirements of Regulation (EU) No. 1227/2011 on wholesale energy market integrity and transparency (also 'REMIT'). 30 This Regulation introduced a new and unprecedented sector-specific legal and monitoring 28 On the other side, transparency can reduce the incentive to invest since the enhanced level of competition can diminish expected profits. Transparency can also support collusion among producers in oligopolistic markets since it facilitates cooperation and free riding detection. Finally, implementing all the necessary informative flows has a cost and this may penalize smaller market players to the advantage of the larger ones (De Francisci, 2014). 29 In 2019, more than 50 DPs, several thousand PDOs and around 12 000 users were registered on the platform, which receives over 10 million files each year (ENTSO-E, 2019a;Hirth et al., 2018). 30 Art. 2(1) of Regulation (EU) No 1227/2011 defines inside information as: 'information of a precise nature which has not been made public, which relates, directly or indirectly, to one or more wholesale energy products and which, if it were made public, would be likely to significantly affect the prices of those wholesale energy products'. framework to detect and prevent market abuse in European wholesale energy markets following the model already applied in the financial sector. It prohibits insider trading, market manipulation and attempted market manipulation, includes obligations for market participants to publish inside information, and requires persons professionally arranging transactions to establish and maintain effective arrangements to detect market abuse and to notify suspicious cases to national regulatory authorities (ACER, 2016). All wholesale market transactions must be reported at EU-level to the Agency for the Cooperation of Energy Regulators (ACER), who carries out its REMIT monitoring duties in close collaboration with national regulatory authorities. To support the transfer of large files and provide an adequate data management environment, ACER launched in 2015 its REMIT Portal, which serves as the single point of access for all REMIT-related applications and key documents. 31 Some intermediaries operate their own IIPs to fulfil REMIT requirements, while others use ENTSO-E's Transparency Platform to publish the required information. 32

Platforms to manage consumer data
During the liberalization of the electricity industry in the 1990s and 2000s there was a general understanding that coordination at the distribution and retail level was less important than at the transmission and wholesale one. This explains the stronger focus on the establishment and regulation of platforms to support and facilitate wholesale trade, as seen in Section 5.1 and 5.2. However, in recent years the deployment of a growing amount of distributed energy resources and the offering of an increasing variety of services to retail customers have been leading to a more complex environment. In addition to the DSO, which is normally in charge of customer connection and metering, and the energy supplier, which is normally tasked with the procurement of energy, balancing and billing of its customers, we are witnessing a multiplication of involved active parties. Interconnected TSOs would like to increase their visibility of what is going on behind the connection point with the distribution network and expand their control on the resources located there in order to improve the secure operation of the whole system. New market participants, often independent from traditional energy suppliers, are willing to access final customers and their data in order to offer them innovative services like demand response, home energy management, P2P energy trading and so on (see Section 6.1 for more on this). Final consumers themselves are increasingly interested in getting more control of their own data in order to fully participate in energy markets and be part of the energy transition.
This evolution of the distribution and retail space multiplies the cases in which access and exchange of consumer data is needed to support the 'decentralized' coordination of the actors populating such space and to ensure an efficient and effective allocation of resources. The establishment of IT systems and digital platforms play a fundamental role in this regard, by streamlining the collection, validation and storage of consumer data (see Section 2.1). Moreover, IT systems and digital platforms can simplify the access and exchange of relevant data and information with eligible parties, thereby allowing the removal of some of the existing barriers to market entry and empowering final customers to share their data based on consent.
The establishment of IT systems and digital platforms to manage consumer data presents a high degree of heterogeneity across countries, even among jurisdictions that pursue the creation of a single market for electricity, as for instance in the EU. This is consistent with a more fragmented and less harmonized organization and regulation of distribution networks and retail markets than is typically the case for transmission networks and wholesale markets. Depending on several factors, as the explicit choices made by policymakers and regulators or simply the enduring legacy of decisions taken in the past, possibly by the incumbent before the liberalization of the industry, different data management 31 At the end of 2019, almost 15 000 market participants were registered with ACER, whose system collected and managed around 1.2 million records in the same year (ACER, 2019). 32 At the time of writing, the REMIT Information System lists 15 dedicated IIPs for electricity (ACER, 2020). models (DMMs) exist and are typically divided into three groups, namely decentralized, partially centralized and fully centralized models, the latter often called 'data hubs' (CEER, 2016). 33 This is not the place to present these alternative models in detail. 34 It is here sufficient to say that in a decentralized model data are typically stored at the source (e.g., metering data are stored by the DSO) and different systems communicate directly with each other through distinct access points and in formats often not standardized (e.g., a supplier receives the information on the energy consumed by its customers from the DSO to which these customers are connected). On the contrary, in a centralized model there is a data hub to which all data are sent and stored. A specific party, not necessarily the one that collects consumer data, runs the hub and operates its functionalities for all the stakeholders, usually in a highly standardized way. Finally, in a partially centralized model only some aspects of data management are centralized while others rely on the direct communication between the various stakeholders (GEODE, 2020;CEER, 2016).
Currently, there is an understanding that each group of DMMs presents its advantages and disadvantages and that there is no-one-size-fits-all solution. Because of that, although several countries like Norway, Italy, Denmark or Spain are moving from a decentralized DMM to a centralized or partially centralized one and although there is a growing consensus that interoperability is an important principle to follow in any case, differentiations in the platforms for managing consumer data are expected to continue in the future (CEER, 2016;CEDEC et al., 2016;Eurelectric, 2016). 35

Empowering final customers and new entrants
The digitalization of the electricity infrastructure paves the way to the emergence of new forms of interaction and exchange among the actors involved in the sector (Glachant and Rossetto, 2018). New products and markets can appear. In the 1990s it was electricity as a commodity that could be traded at the wholesale level. Today, with the sprawling of digital technologies to the level of distribution grids and retail-size assets, it is a whole new set of energy services and tailor-made products that address the specific needs and preferences of customers.
Although not yet dominant, innovative data-driven and often asset-light business models offer opportunities to new entrants and empower smaller energy customers like households and small businesses. Traditional actors like energy suppliers or grid operators can be disrupted and may need to reinvent themselves if they want to survive and preserve their salience in the electricity sector of the future.
In this section, we present the more transformative implications that digitalization has for the electricity sector. Section 6.1 explains how the availability and usability of data generated by the digitalization of the electricity infrastructure promote the appearance of innovative value propositions, possibly introduced by new players, which expand the options and expectations of final customers. Section 6.2 outlines how these novelties represent a challenge for the traditional managers of the electricity infrastructure, both in terms of generation and grids, who may nonetheless react and try to transform this challenge into an opportunity. 33 A data management model refers to 'the framework of roles and responsibilities assigned to any party within the electricity system and market and the subsequent duties related to data collection, processing, delivery, exchanges, publishing and access' (CEDEC et al., 2016, p. 11). 34 The interested reader may find additional information in .

From the supply of kWh to the provision of energy as a service
The increased availability and usability of detailed data concerning the energy consumed or produced by final customers allow the creation of innovative products and pricing models (Brown et al., 2019;Glachant, 2019;CEER, 2019b;Morris et al., 2020). Beyond the traditional supply of kWh at a predefined price and connection point with the grid, the provision of tailor-made services addressing the specific needs and preferences of each customer, be she a household or a small factory, becomes possible.
These services can take the form of more or less sophisticated energy insights that tell the customer about where and when she consumes how much energy, how she can reduce her consumption by changing her behaviour or how she can save money by switching to a different tariff or supplier. They can also consist in the provision of electricity with particular and guaranteed attributes like its generation at the local level or from renewable sources. Or they can involve the provision of energy-as-a-service (EaaS), where the customer does not buy a certain amount of kWh but rather a certain level of energy service or comfort (e.g., the temperature in a room or the number of km travelled), 36 leaving to the provider the decision on how to deliver such service or comfort level (IRENA, 2020c). 37 In this case, the provider may choose to equip the customer with more efficient appliances or with a distributed generation unit like a PV panel. The provider may also decide to equip the customer with a home energy management system (HEMS), which brings together all the major energy assets located at the customer premisese.g., heat pumps, batteries, EV chargers and so onand operates them intelligently with the goal of minimizing the cost of the overall energy service, while respecting the conditions agreed with the customer. 38 Other possibilities exist as well. A customer, for instance, can accept to reduce further her control on the energy assets located at her premises and allow the service provider to use them to sell energy and ancillary services back to other market parties or to the system operator. 39 In this case, the demand response implemented by the assets of the individual customer is normally aggregated with that of hundreds or thousands of other individual customers in order to achieve a sufficient scale and be able to offer efficiently on the wholesale market or to meet the minimum requirements defined by the system operator (IRENA, 2019e). In exchange for the reduced level of control, the customer receives from the service provider, which in this case is called an aggregator, a share of the revenues generated by the sale of energy or other services. 40 Innovative services can also take the form of platforms for the direct exchange of electricity or other attributes peer-to-peer (P2P). In this case, the customer pays for access to the platformeither via a 36 For more information on mobility-as-a-service, consider Chapter 4 in this book. 37 The emergence of some of these new services fosters the integration of different energy vectors and different final uses.
When a customer purchases EaaS, for instance, the focus is no more on a specific energy vector like electricity or natural gas, but rather on the final use (transport, heating and cooling, and so on). And when the customer accepts solutions like HEMS, then all the various vectors and uses are operated holistically in order to minimize the overall energy use or cost. This is the essence of sector integration or convergence. 38 According to the IEA, the digitalization of buildings can reduce their energy consumption in 2040 by 10% with respect to the baseline forecast. Particularly relevant in this regard are solutions for the smart provision of heating and cooling. Smart thermostats and heat-as-as-service offers could reduce energy consumption related to thermal needs by between 15 and 50% (IEA, 2017, pp. 42-46). More generally, existing solutions delivering EaaS claim to provide significant reduction in energy billsbetween -15 and -40%and peak demandbetween -3 and -10%already today (IRENA, 2020c, p. 13). 39 Similarly, the service provider can take some form of control over the EV of its customer, or over the fleet of EVs with which it provides mobility-as-a-service, in order to use the battery of the EV for the delivery of grid services to the system operator or to arbitrage on the wholesale market for energy. This type of business models is often called vehicle-to-grid (V2G) and is made technically possible by so-called smart charging solutions. 40 In essence aggregators allow retail to 're-enter' wholesale as an offer to balance the system or satisfy the needs of unbalanced market parties. By doing this, aggregators perform the reverse function of normal energy retailers. Both independent and integrated aggregators exist (Glachant, 2019). Robert Schuman Centre for Advanced Studies Working Papers periodic subscription fee or via a small transaction-related chargeand gets the possibility to interact with other customers without having to take care of all the intricacies of energy markets (Glachant, 2020). This type of services can target individual customers or, alternatively, communities of customers that want to act in the energy field together and that might ask professional entities for support in dealing with the more sophisticated activities, as for instance the sharing of electricity over the public network or the creation and management of a common mini-grid located at the edge of the public infrastructure.
The development of these new services addresses the growing stratification and increasing expectations of final customers (Sioshansi, 2019). While in the past final customers, especially those in the retail segment, were quite homogeneous and had limited expectations on the way they could interact with the electricity supplier, the situation today is becoming quite different.
First, final customers face a growing number of options and are getting more heterogeneous in terms of energy-related needs and preferences. Some customers may be more environment-friendly, while others may be more interested in getting the cheapest offer for the electricity they consume. Some customers may have much higher electricity needs, as for instance the early adopters of electric vehicles, while others may have limited consumption because they are rarely at home. Some may own their house and have the resources to invest in distributed generation and batteries, while others may be renting a small flat for a short period of time and be budget-constrained. Digitalization allows to consider these different situations and develop targeted propositions that provide a larger value to different types of customers.
Second, in the purchase of several goods and services final customers have become used to a superior customer experience than in the past, normally characterized by simplicity, ease of use, transparency and control over a wide range of options. They now expect, or at least wish, that the purchase of energyrelated services shares the same characteristics and is not much different from booking an accommodation or an airline ticket. Again, digital solutions can contribute to the fulfilment of customers' expectations in energy in ways that were hardly imaginable a few years ago.
The roll-out of smart meters, the development of AI-based predictive tools, the implementation of IoT solutions and the like do not only empower final customers but also offer opportunities to new players (Sioshansi, 2017;Sioshansi, 2019;. Companies able to deal with the data layer generated by the digitalization of the electricity infrastructure, both in front of and behind the meter of final customers, can enter the sector without the need to invest large sums of money in any physical asset. 41 In most of the cases mentioned above, innovative companies simply have to develop the software necessary to process the relevant data and ensure a better customer experience. 42 They may need to partner with technology vendors or with installation companies that deploy the necessary hardware at the customers' premises, but most of the time they do not have to invest by themselves in those activities. The smart use of the data produced by the digitalized assets and the new forms of interaction with customers are sufficient to permit them to package new products and differentiate their offer from that of traditional energy suppliers. 43 Nevertheless, the fact that barriers to entry are lower does not mean that life is easy for new players. In the second decade of the twenty-first century, we saw a proliferation of start-ups and non-energy 41 An example of business model without physical assets is given by Tomorrow, a European technology start-up that provides information on the marginal CO2 content of the electricity consumed in a certain country during a certain hour. Tomorrow provides this service via an application called 'electricityMap' that relies on the data of the ENTSO-E Transparency Platform (Corradi, 2019). 42 Innovative market players may be primarily interested in the acquisition and monetization of consumer data. In this case they could be willing to sell their energy-related services to customers at a loss. 43 Product differentiation has been traditionally difficult in the electricity sector. Electricity per se is a homogeneous product and given the fact that distribution is a regulated activity, conventional suppliers can compete only in terms of price or better customer management (commercial quality of service).
European University Institute 23 companies developing new services and products in the energy field (Küfeoğlu et al., 2019). Some of them have raised significant attention and financing, including from some of the traditional energy players, but in many cases they have not been able to scale and get out of their initial niches. There are several reasons for these difficulties: customer acquisition takes time, especially for new firms without a recognized brand; some products still require the installation of specific assets whose cost-recovery must be ensured by long-term contracts that customers are often reluctant to sign; the integrated management and optimization of multiple assets necessitates full interoperability and this may be missing due to different choices by technology vendors in terms of protocols and interfaces; access to customer and network data may be difficult or only partial; network tariffs and energy markets regulation may limit the monetary value of certain propositions and reduce incentives for customers to accept a certain offer. 44

The impact on the traditional infrastructure managers
The empowerment of final customers and new entrants challenges the traditional owners and managers of the electricity infrastructure, both (utility-scale) generation and networks (Helm, 2017). Generators, which are often also suppliers of electricity, are challenged because digitalization increases the observability, predictability and controllability of any asset connected to the electricity network, both on the demand and supply side. Even the smallest consumption and generation unit, as for example a home battery or a residential heat pump, can be monitored and operated with the goal of minimizing not just its individual energy use, but rather the cost of providing energy services to the whole house or to the community to which that house belongs. This bottom-up approach to resource optimization is further empowered by the great reduction of transaction costs enabled by digitalization and the emergence of new forms of intermediations that foster coordination among customers or even among devices (Tirole, 2017). As a consequence of that, where population or the level of economic activity remain the same, demand of electricity from the public grid could stagnate or even decline over time (Fox-Penner, 2020). 45 The value of owning a utility-scale power plant and producing pure kWh is likely to diminish and be subject to additional long-term uncertainty. Moreover, companies generating and retailing electricity, the so-called 'gentailer', may lose, at least in the eyes of their customers, their current role of main interface with the energy system. Empowered by digitalization, new service providers and platforms may try to supplant such role and reduce the supplier to a mere provider of back-up energy for the moments in which production and storage at the customer premises or energy sharing with other members of the community to which she belongs are not enough to balance consumption. 46 The owners and operators of electricity networks are challenged by digitalization as well, since the developments anticipated in Section 6.1 are likely to lead to a more variable and less predictable use of networks. Although customers may occasionally rely extensively on the public gridfor instance during cold winter days when the output of rooftop PV panels is limitedand full grid defection is hardly an economically sensible option in countries with a developed infrastructure in place, it is probable that the increased deployment of distributed generation, distributed storage, smart solutions for active load management and mini-grids will lead to a reduced amount of electricity that is, on average, withdrawn from or injected into the public network. This reduction in the load factor, which can originate a decline 44 As a result, only a few energy start-ups have prospered, as for instance Next Kraftwerke, an aggregator, and Octopus Energy, an innovative retailer. Some successful new players have been acquired by conventional electric utilities willing to innovate and embrace change, as for instance the aggregator EnerNOC bought by Enel, or by oil and gas companies willing to enter the electricity sector, as for instance Sonnen, a producer of smart home batteries and energy retailer, recently purchased by Shell. 45 Electrification of transport and heating and cooling may provide significant new sources of electricity demand growth.
However, it is unclear whether this will translate into net demand growth of electricity from the public grid. 46 Suppliers may not only see a reduction in the amount of electricity they sell, but also an increase in their costs due to the augmented complexity for them to balance the net position of their customers. This point has been raised in particular with regard to the appearance of independent aggregators. See for a recent review of the issue Schittekatte et al. (2021). Robert Schuman Centre for Advanced Studies Working Papers in revenues for grid companies if network tariffs are mainly based on the amount of kWh that pass though the network, may take place in a moment in which the same grid companies have to replace their aging infrastructure and expand their capacity to ensure that grids can adequately and reliably deal with all the requests by network users (Monitor Deloitte et al., 2020). Nonetheless, the challenges for grid companies are not only related to this possible combination of revenue erosion and increasing costs, but also to the likely augmented complexity of ensuring the continuous and secure operation of their systems. If coordination with the new service providers and intermediaries is not guaranteed, it may happen that the energy flows steered by them produce imbalances or congestions in the grid that are difficult for system operators to anticipate and manage. In Europe and the US, this problem emerged very clearly at the transmission/wholesale level during the liberalization period and was addressed with the definition of detailed network codes that market parties and network users must respect. With the appearance of energy communities, platforms for P2P energy trading and the like, similar issues are bound to become more relevant at the distribution/retail level too.
However, digitalization is not only a challenge but also an opportunity for the traditional owners and managers of the electricity infrastructure. These actors can choose to embrace the change and leverage on their existing strengths to react to the entrance of new players and benefit from the digitalization of the electricity infrastructure. Doing this is not easy, as it entails the acquisition of new skills, the development of new practices and probably a deep change in corporate culture, but it is possible, as some (so far) successful cases of forward-looking electric utilities show.
Integrated generators and suppliers may adopt a more 'customer-centric' approach, where they stop looking at the number of kWh sold and increase their focus on the needs and expectations of their customers. By taking advantage of their existing customer base and the related data and revenue flows, they may partner with technology vendors, installers, software companies and the like to swiftly offer a comprehensive and targeted set of energy services to a large audience. Instead of providing simply electricity, they can become a one-stop-shop that smoothly satisfies all the energy-related needs of their customers, from the replacement of less efficient energy appliances to the installation of PV panels for self-consumption.
Grid companies can equally react and try to benefit from digitalization by increasing their role of 'neutral market facilitators' (EY and Eurelectric, 2019;CEER, 2019a;GEODE, 2019). This is particularly the case for distribution companies which lie at the centre of most of the developments discussed in Section 6.1. In addition to the provision of the hardware that supports the interactions at the distribution/retail level, DSOs, where compatible with unbundling rules and the preservation of an undistorted competitive level playing field, could offer a series of services to their network users and to market parties. The roll-out of smart meters represents an important development in this regard that has simplified, where it occurred, existing activities by DSOs and offered them new opportunities. By providing the basic inputs for the development and implementation of commercial applications by third parties, DSOs could address the risk of being 'platformed' by the emergence of new digital intermediaries (Montero and Finger, 2017) and, by turning themselves in a kind of platform, develop new revenue streams (Zarakas, 2017;Peterson and Ros, 2018).
Grid companies can also benefit from the activation of retail-size units and the appearance of innovative players by getting access to an increasing amount of flexible resources and providers of ancillary services, both at the transmission and distribution level. By mobilizing them, grid companies could improve their ability to operate the electricity system and defer costly investment in additional physical capacity. Again, this change would be particularly relevant for DSOs, who, until now, have rarely been used to implement so called non-wire alternatives to grid expansion (CEER, 2019a;IRENA, 2019d). Currently, local flexibility markets are being tested in various countries across Europe with the direct involvement of DSOs. Several options seem to be available in this regard . DSOs, for instance, may partner with innovative start-ups or collaborate with existing power exchanges willing to develop new marketplaces and services; they may also act alone or in close cooperation with the interconnected TSO and other DSOs.

Challenges for the organisation and regulation of the sector
The wide application of digital technologies in the electricity sector profoundly expands the boundaries of what is possible. As this chapter shows, these technologies enable a more granular and closer to real time observation of the electricity infrastructure, a more reliable and dynamic prediction of its future functioning and usage, and an enhanced and more effective control of its components. Digital technologies allow to monitor and measure the actions of the actors using the electricity network, to share the collected data among different entities and to achieve a more efficient employment of the available resources. All of this implies, by adopting the jargon of institutional economics, the reduction of transaction costs and the possibility to define, allocate and enforce new property rights. These changes question the existing organization and regulation of the sector and the opportunity to adopt alternative and more efficient institutional arrangements (Brousseau and Glachant, 2008). This happened already in the 1980s and 1990s, when the first wave of digitalization targeting large power plants and transmission lines significantly reduced the cost of collecting and processing real-time information on injections and withdrawals of power, paving the way, together with other factors like the development of combined cycle gas turbines, to the establishment of the early competitive wholesale markets for electricity. Something that had been deemed impossible since the very beginning of the industry in the late nineteenth century progressively turned out to be technically feasible and compatible with the efficient supply of electricity to final customers.
The more recent waves of digitalization targeting distribution grids and the premises of final customers question again the organization and regulation of the sector. Marginally touched by digitalization 30 to 20 years ago, the distribution and retail spaces were at that time not always considered essential for the development of competitive and efficient electricity markets. Some scholars considered the restructuring of the retail activity as providing limited or no benefit to final customers and judged the unbundling of distribution grids from generators and suppliers less urgent than that of transmission networks. Reforms around the world were not universally characterized by the introduction of competition in the retail segment, especially after the California energy crisis of 2000-01 (Littlechild, 2021). And even where retail competition was consistently pursued, as for instance in the EU, this occurred gradually and in a context of limited options and close oversight. Customers were allowed to choose a different energy supplier, but many aspects of their relationship with it were strictly defined by regulation, including the prohibition to have more than one supplier per connection point (Poudineh, 2019). 47 Similarly, in most of the countries in which the industry was restructured, only accounting or legal unbundling was normally required. Full separation of distribution from competitive activities was rarely implemented, the most notable case in the EU being the Netherlands.
The emerging perception is different nowadays. The digitalization of smaller assets and the activation of retail customers blur the distinction between the transmission and wholesale level on the one hand and the distribution and retail level on the other. What 30 years ago could be seen as quite separate 'modules' of the electricity sector, with quite different characteristics and needs for efficient organization, are now appearing as more similar to each other. Governance issues that were identified at the transmission/wholesale level are now apparent also at the distribution/retail one and the interface between the two is becoming more complex and multifaceted. To further complicate the situation, the policy objective of an accelerated transition to a low-carbon energy system, which is largely shared around the world, makes these transformations and the relative challenges even more pressing and demanding.
Discussion of these issues and challenges, both at the academic and institutional level, is ongoing (MIT, 2016;Burger et al., 2019a;Burger et al., 2019b;CEER, 2019b). A comprehensive and detailed analysis is beyond the scope of this chapter. In what follows, we sketch out the most relevant aspects raised by recent waves of digitalization and some of the conclusions on which a consensus on how to fully exploit the potential of digitalization is gradually emerging. These aspects and conclusions can be grouped around four ideas: 1) data become a fundamental productive input; 2) the distinction between transmission and distribution is blurring; 3) the appearance of new products and market players requires the reassessment of rules developed for a previous era; 4) regulators face a more complex and uncertain environment calling for an upgrade in the approach to regulation.
First, in a digitalized electricity sector data become a fundamental input for the efficient and effective delivery of both existing and still to be defined goods and services. Although the comparison may be somewhat stretched, data, or at least some categories of them, can be considered an essential facility, as the electricity grid normally is. And like the electricity grid, they must be generated and made available and usable by all the pertinent stakeholders in a secure, transparent and non-discriminatory manner. 48 This requirement expands beyond wholesale market and transmission network data, which are already collected and normally accessible to the public in many liberalized electricity systems (see Section 5.2). This requirement extends now to distribution network and consumer data. The roll-out of smart meters for retail customers is in this respect an important first step, but it is far from being sufficient. Accurate registries of connected assets, especially DG units and EV charging points, could be necessary as well, in order to increase the visibility of the distribution network. All these data should then be open not only to the system operator but also to any market party. Of course, some concerns in this regard are legitimate. Accessibility to detailed network data may create security issues, as it may enable criminal or terrorist activities targeting critical infrastructures, and even privacy concerns, as it may facilitate the identification of personal consumption patterns. For these reasons, there is an evolving understanding that there should be some limit in the disclosure of network data and that consumer data should be accessible by third parties only with the direct consent of the relevant consumer or only if properly aggregated and anonymized (CEER, 2019a).
Second, in a digitalized electricity sector the distinction between transmission and distribution tend to fade away. Active network management becomes possible and in fact necessary on both levels, especially to support an accelerated decentralization and decarbonization of the energy system at the least cost. DSOs, which are currently mostly tasked only with the building and maintenance of distribution networks, must turn into proper system operators that deal with local congestions and other network issues by procuring ancillary services. At the same time, network users, especially those connected at the distribution level, should explore the opportunities offered them by new technologies to make more efficient use of the electricity grid. Profound transformations of the existing regulatory framework for grids are required to support these changes. TSOs and DSOs must be incentivized to resort, whenever more efficient, to non-wire alternatives, as for example the market-based procurement of flexibility from DERs. This means the removal of any bias in favour of capital expenditure in the economic regulation of networks, and the establishment of more open and transparent markets for the provision of services to system operatorsincluding at the local levelin which smaller assets, also on the demand side, can participate. A reassessment of unbundling rules is likely to be necessary in this regard, in order to ensure that DSOs do not get involved in (potentially) competitive activities or distort competition by providing undue advantage to any company related to them. 49 With DSOs becoming active managers of their grids, vertical coordination with the interconnected TSOs and other DSOs becomes extremely important in order to ensure consistent and efficient choices in system planning and operation. Initiatives in this area are visible, for instance in the EU (CEDEC et al., 2019), but there is 48 Some form of standardization and data interoperability seem to be key, in particular in order to promote a more competitive and dynamic retail market. 49 The attribution of data management responsibilities to DSOs raises some concerns in this regard. While DSOs could perform well those responsibilities thanks to their monopoly position and direct access to vast amounts of consumer and network data, the lack of full separation from companies performing commercial activities may create conflicting interests (Buchmann, 2017). Similar arguments are provided by Buchmann (2020) with reference to the establishment of local congestion markets. On a different position, there is Nillesen and Pollitt (2021). not yet a shared vision on who should do what and how to coordinate system operation at different levels (Hadush and Meeus, 2018;CEER, 2019b). Finally, in a world where customers have more options than in the past with regard to when and how much energy to consume, produce or store, more detailed economic signals must be provided to them in order to promote a more efficient use of the electricity infrastructure, ensure cost recovery for network companies and avoid an unfair and inefficient shifting of network costs towards less active electricity consumers (CEER, 2019b;Nouicer et al., 2020). This move towards a more granular and cost-reflective network charging questions one of the traditional linchpins of electricity regulation, that is the provision of network access at the same price for all final customers located at the same voltage level, but it might well be that in a digitalized electricity sector a more differentiated treatment is necessary to ensure that everybody is treated fairly.
Third, in a digitalized electricity sector new actors and intermediaries quickly emerge and are able to offer new products to final customers and grid operators alike (see Section 6.1). In this context, the allocation of rights and duties to the various market players must be reassessed in order to ensure that they all face a level playing field and innovation is not stifled. In electricity, like in any other industry, rules rarely have universal value; on the contrary, they are often a legacy of the past. They were 'manufactured' in a certain way because at a certain moment they looked adequate and acceptable to manage certain interactions and provide certain sufficiently efficient outcomes; however, with the evolution of technology, needs, preferences and business models, they may well become obsolete and stop being fit for purpose (Brousseau and Glachant, 2014). In electricity, some pieces of regulation could suffer from this kind of 'aging' because of digitalization, and may require a reassessment. An example is the rule of the single supplier and its role as hub for the implementation of all the transactions with the final customer. This rule looked reasonable and even 'natural' in the 1990s, when the possibility of monitoring granularly the consumption of energy by a final customer, especially a residential one, was limited, the processing, storing and sharing of data were expensive, and the availability of these consumer data was of little value. Today, monitoring energy flows at the customer level is much easier, processing, storing and sharing the relative data have become much cheaper, and the potential uses of those data have significantly expanded, often in ways that could not have been anticipated. All of this questions the prohibition for final customers to contract with more than one supplier at the time and opens the door to the adoption of new rules that can support a better exploitation of the potential of digitalization (Ofgem, 2017b).
Fourth, in a digitalized electricity sector, changes in technology occur faster than before and consumers become more heterogeneous. In these conditions, pursuing the protection of energy consumers and an efficient and sustainable organization of the sector gets more difficult and is likely to require a transformation in the way regulators approach their duties and arrange their activities (Glachant, 2012). New skills and competencies in ICT, big data and behavioural sciences are certainly a starting point, but are far from enough to manage raising complexity and uncertainty (CEER, 2019b). A close monitoring of market developments and the promotion of open 'regulatory fora', where stakeholders are invited to participate and are induced to reveal information about new technologies and business models, represent essential initiatives to deal with a constantly evolving and unbridgeable information asymmetry that penalizes regulators (Brousseau and Glachant, 2011;CEER, 2019b). Cooperation with regulators acting in other sectors is important too, since digitalization is sometimes more advanced there and lessons can already be learnt; on top of that, cross-sectoral approaches can be particularly useful when bundled products going beyond the provision of pure electricity must be investigated (CEER, 2019b;Morris et al., 2020). However, given the hardly predictable implications of certain innovations and the possibility that some promising routes are not pursued due to existing rules or excessive risks, regulators should not only monitor innovation, but also actively foster it. This can happen in several ways. Regulators can finance research and development projects via dedicated programmes funded by ratepayers or the general state budget; they can provide an extra-remuneration to regulated network companies that deploy innovative and not yet fully mature technologies; or they can provide targeted exemptions from existing rules in order to experiment with some specific activities Robert Schuman Centre for Advanced Studies Working Papers or solutions that would otherwise not be possible to perform . More generally, regulators should be willing to adopt a dynamic approach to regulation. Agility and adaptability should become important principles framing their work in a fast-evolving sector. This means to be ready to make rapid changes if a certain rule is not fit for purpose anymore or is actually blocking useful innovation (CEER, 2019b). Ultimately, the digitalization of the electricity sector is likely to require regulators changing their attitudes towards electricity consumers and their view on how to protect them (Ofgem, 2017a;CEER, 2019b). Two decades or more after the opening of retail markets for electricity and in the face of the large opportunities offered by digital technologies, a sector-specific and uniform consumer protection based on a traditional interpretation of the concept of universal public service could be outdated. It may be replaced by an approach that prioritizes the empowerment of consumers and puts a dedicated focus on those who are not (yet) able to adequately engage with new technologies and service providers (for instance those suffering from digital divide or with limited financial resources). Addressing those vulnerable consumers who face a concrete risk of being worse-off, while enabling the others to explore new products and interact with new intermediaries and service providers looks like the necessary step ahead for the regulation of a digitalized infrastructure that supports the decentralization and decarbonization of the power sector.

Conclusions
Digitalization is not a novelty for electricity but a process that has come in successive waves. It is the result of the application of information and communication technologies to the different elements of the physical infrastructure, but its implications go much beyond technological change. Like in other industries, digitalization allows to generate, transmit and analyse data, and to implement control on the physical infrastructure with a speed and reduction in costs that are extraordinary when compared to the previous analogue world. Over time, digitalization has led to the emergence of a proper data layer on top of the physical one, covering different domainsnetworks, markets, consumers and external datawhich are characterized by a heterogeneous level of development and governance.
The abundance of data generated by digital technologies and the possibility to process and use them cheaply and quickly imply first of all a series of efficiency gains in terms of infrastructure planning, operation and maintenance. This applies to both electricity generation and network assets. Investments in new physical capacity can be planned in a smarter way, ensuring a higher probability that its use will be maximized. Similarly, the operator of a 'digitalized' asset, be it a gas-fired turbine or an electric line, can be constantly informed of the status of the asset and estimate, with the support of digital twins and artificial intelligence, when maintenance is likely to be needed. A swifter detection of faults and the implementation of automated self-healing procedures are equally positive consequences of digitalization for the maintenance of the electricity infrastructure.
The increased observability, predictability and controllability of physical assets enable a more efficient operation of existing capacity. This is particularly significant for electricity networks whose use must respect stringent security conditions imposed by the laws of physics. Thanks to digitalization, transmission and distribution system operators can better understand how much electricity can flow over a line and react to contingencies. As a result they can offer more capacity to network users without facing higher risks. At the same time, digitalization enables system operators to gain a more granular understanding of how network users are 'utilizing' their grid, provide them with more accurate signals to steer demand for capacity off-peak, and achieve a higher load factor. The penetration of variable renewable energy sources and distributed generation in the electricity mix further increases the importance of these positive implications of digitalization for network operation.
By decreasing the cost of data generation, reproduction and sharing, digitalization expands the possibility to coordinate the different actors that populate an increasingly decentralized electricity sector. After the liberalization and vertical disintegration of the industry that occurred in many countries around the end of the twentieth century, platforms have started to play a fundamental role in the creation of new marketplaces, where products like electricity, transmission capacity or reserve are defined, sellers and buyers meet and agree on transactions, and the consistency of those transactions with system operation is assessed. Platforms also support coordination by gathering and making available to all the stakeholders a large amount of data on the condition of the physical infrastructure, the operation of electricity markets and the final consumers.
Digitalization does not only increase efficiency in performing existing activities but also enables new forms of interaction. In the 1990s it played an important role in the establishment of competitive wholesale markets for electricity. Today it empowers final customers and new entrants by enabling the development of new services and asset-light business models, particularly in the distribution and retail spaces. Traditional infrastructure managers, both utility-scale generators and network operators, can face disruption. The direct control of physical assets, for instance, is less relevant in a world where data are abundant and usable, the provision of customer-focused services may replace the supply of kWh, and the use of the public network can be significantly reduced thanks to the optimization of DERs. Nevertheless, the death knell has not sounded yet for traditional infrastructure managers. They still benefit from significant advantages, including their large customer base or their legal monopoly on system operation. Building on them, traditional infrastructure managers can react and reinvent themselves to ensure their continued salience in the electricity sector of the future.
Digitalization of the electricity infrastructure has been transforming the sector since decades. By reducing transaction costs and enabling the definition, allocation and enforcement of new property rights, it questions the existing organization and regulation of the sector, and suggests the opportunity to adopt alternative and more efficient institutional arrangements. This has become particularly apparent regarding electricity distribution and retail. Marginally touched by digitalization until a few years ago, distribution and retail are now at the centre of change and are likely to play a fundamental role in supporting the rapid and cost-effective transition to a low-carbon economy. In this context, policymakers and regulators are called to assess and, if needed, amend some of the existing rules or create new ones concerning, for example, data management, incentive regulation and unbundling of DSOs, TSO-DSO coordination, and the rights and duties of energy suppliers. The tasks of policymakers and regulators are not easy due to the complexity and uncertainty that stem from an accelerated technology development and a growing customer differentiation. Therefore, a more dynamic and less conservative approach to the regulation of electricity seems required to secure the full exploitation of the potential of digitalization.