All three layers encrypt with the same algorithm, AES-256. They differ in what exactly they hide, how much throughput they take, and what latency they introduce. The spread is wider than the apparent similarity suggests, because it ranges from nanoseconds and zero overhead in the optical layer to hundreds of microseconds and a clear loss of throughput with IPsec.


The choice is not a matter of preference or of a level of security. It is a consequence of which route you are protecting, what your latency requirements are, and where the trust boundary runs in your network. Below is a comparison of the three layers together with three deployment scenarios and an assessment of the post-quantum readiness of each.

The scope of protection on a single route


IPsec protects the whole route, but the data is in the clear inside both edge routers. MACsec protects each hop separately, so at every node the frame is decrypted and encrypted again. Layer one encryption covers only the optical path, but with no overhead and without exposing metadata.

Each layer protects a different section of the route

The same path, three different scopes of protection
Router A
Node A
Node B
Router B
Layer 3 · IPsec end to end, IP headers visible
encrypted along the whole route, clear in the routers
Layer 2 · MACsec separately on each hop, data clear in the nodes
hop
hop
hop
Layer 1 · OTNsec optical path, zero overhead, metadata hidden
black fibre

IPsec protects the whole route, but the data is in the clear inside both edge routers. MACsec protects each hop separately, so at every node the frame is decrypted and encrypted again. Layer one covers only the optical path, but with no overhead and without exposing metadata.

Three layers, three different scopes of protection

Layer one: optical encryption, OTNsec


Physical-layer encryption operates at the level of the OTN frame, before the signal is sent into the fibre. The frame payload is encrypted one to one, with no bytes added. The authentication data fits into reserved fields of the OTN header, and the key exchange uses an auxiliary channel that is part of the frame anyway. The consequence is full protocol transparency. Everything that passes through the path is encrypted, regardless of whether it is Ethernet, Fibre Channel, OTN, or something else. To an observer tapping the fibre, neither addresses nor traffic patterns are visible. In industry materials this is referred to as black fibre. The limitation is just as clear, because the solution requires dedicated hardware on both sides of the link and works point to point. It cannot be stretched across a route with another vendor's device in the middle.

Layer two: MACsec


MACsec is the IEEE 802.1AE standard, encrypting Ethernet frames between adjacent devices. The overhead is about 32 bytes per frame, made up of the SecTAG header and the ICV integrity check value. Encryption is done in hardware, hence the low latency. The scope of protection is a single link section, that is a hop. When data passes through several layer-two devices, each of them decrypts the frame and encrypts it again, so inside every device along the way the data exists in the clear. The MAC headers remain visible, because they are needed for frame switching.

Layer three: IPsec

IPsec protects IP packets end to end. Intermediate devices have no way to decrypt them, which makes it the foundation of VPN solutions and the only sensible option when traffic passes through infrastructure you do not control. The price is complexity and performance. Processing is often done in software, the header overhead is markedly larger than with MACsec, and the latency rises by one or two orders of magnitude. In tunnel mode the outer IP headers remain visible.

The comparison


The encryption algorithm is the same in all three cases, usually AES-256. The difference in security therefore does not stem from the strength of the cipher, but from how large a portion of the communication a given layer covers and over what section.

Criterion Layer 1OTNsec Layer 2MACsec Layer 3IPsec
Per-frame overheadzero, 1:1 mappingabout 32 bytestens of bytes, depending on the mode
Effective throughputfulla small lossa significant loss, dependent on packet size
Latencyon the order of nanosecondssingle microsecondshundreds of microseconds
Scope of protectionthe whole payload including metadataEthernet frame, MAC headers visibleIP packet, outer headers visible
Reach of protectionpoint-to-point in the optical pathhop-by-hopend to end
Protocol transparencyfull, any protocolEthernet onlyIP only
Hardware requirementsdedicated hardware on both sidesa switch or router with 802.1AEbroad, including software
Operation over foreign infrastructurenolimitedyes
Key management complexitylow, auxiliary channel in the framelow, MKAhigh, IKEv2 and policies

Two rows deserve a comment. The scope of metadata protection tends to be overlooked in comparisons, yet in risk analysis it often weighs more than the confidentiality of the content itself. Layer one encryption also hides traffic patterns, because an observer does not know who is communicating with whom or when the intensity rises. MACsec and IPsec leave the MAC addresses and the outer IP addresses respectively, from which the topology and activity can be reconstructed. The reach of protection decides where data exists in the clear. With MACsec every intermediate layer-two device decrypts the frame and encrypts it again, so if the threat model includes those devices, MACsec is not the answer.

Three deployment scenarios

Network operator

An ISP with a backbone network

Links of 100G and above, routes through areas without full physical control. Every percent of throughput is commercial capacity, the traffic of many customers in different protocols, the threat being physical access to the fibre.

Recommendation
Layer 1
OTNsec on the backbone, MACsec at the interfaces with business customers
A bank, two data centres

Synchronous replication

Two sites, distance a dozen to a few dozen km, leased dark fibre. Every microsecond lengthens the write-acknowledgement time, the auditor asks whether the data is in the clear anywhere between the sites.

Recommendation
Layer 1
OTNsec on the route, IPsec as a supplement for selected applications
Local government

A municipal network

The office, schools, subordinate units, monitoring. Mixed traffic, distances of a few to a dozen km, a limited budget, infrastructure often owned, traffic entirely Ethernet.

Recommendation
MACsec
the base layer, OTNsec on selected critical sections

Scenario one: a network operator


An ISP with its own backbone network, links with a throughput of 100G and above, routes running through areas over which the operator does not have full physical control. The recommendation is layer one. The justification has three pillars. First, at a throughput of 100G and above, every percent of lost throughput translates directly into the commercial capacity of the link, so zero overhead stops being a technical detail and becomes a revenue item. Second, the operator carries the traffic of many customers, often in different protocols, and protocol-transparent encryption handles everything without distinction, whereas MACsec would require all the traffic to be Ethernet. Third, the main threat vector here is physical access to the fibre outside the nodes. The methods of tapping include both inserting a splitter and intercepting the signal through an appropriately bent fibre, where a fraction of a percent of the optical power is enough to read the transmission. Layer one encryption protects exactly that section. MACsec remains a sensible supplement at the interfaces with business customers, where the traffic is Ethernet and the requirements concern a single access link.

Scenario two: a bank and two data centres


A financial institution with two processing sites, synchronous replication between them, a distance of a dozen to a few dozen kilometres, leased dark fibre. The recommendation is layer one, with IPsec as a supplementary layer for selected applications. Synchronous replication is the deciding factor here. Every microsecond of latency on the route lengthens the write-acknowledgement time, and with synchronous replication that time translates directly into the performance of the transaction systems. Latency on the order of hundreds of microseconds, characteristic of IPsec, is unacceptable in this scenario as the sole protection for the route. The second factor is throughput, because replicating production environments generates traffic for which a throughput loss on the order of a dozen or so percent means having to buy additional capacity. The third is the regulatory requirement. Requirements for protecting data in transit are usually met by AES-256 encryption regardless of the layer, but auditors also ask whether the data exists anywhere in the clear between the sites. With layer one encryption the answer is no, without reservation. IPsec remains useful for individual applications with heightened requirements, where end-to-end protection must also cover the interior of both sites. It is a supplement, not an alternative.

Scenario three: a local-government unit


A municipal network connecting the office, schools, subordinate units, and monitoring. Mixed traffic, distances of a few to a dozen kilometres, a limited budget, infrastructure often owned. The recommendation is MACsec as the base layer, layer one on selected critical sections. It is the only one of the three scenarios in which MACsec comes out best, and it is worth understanding why. The entry cost is markedly lower, because MACsec is supported by many access and aggregation switches as standard, without the need to buy dedicated encryption devices. In a network with a few dozen nodes the difference in capital outlay is significant. The traffic is entirely Ethernet, so limiting MACsec to one protocol is not a problem. Latency requirements are lenient, because a few microseconds on a section do not matter for line-of-business systems, mail, or video monitoring. There remains the question of sections where traffic passes through areas outside the unit's control, for example a route leased from an operator or running across private land. There it is worth considering layer one encryption on that specific section, not across the whole network. A separate note concerns compliance. The requirements arising from the data-protection regulations and from the national cybersecurity system speak of adequate technical measures, without indicating a specific layer. MACsec deployed consistently on all links meets these requirements, provided a risk analysis is documented that justifies the choice.

Post-quantum readiness


This question comes up today in most procurement proceedings, and it is worth answering precisely, because a fair amount of misunderstanding circulates around it. The symmetric cipher itself is not the problem. AES-256, used in all three layers, is regarded as resistant to known quantum algorithms. Grover's algorithm halves the effective key strength, which at 256 bits still leaves a security level corresponding to 128 bits, and that is a sufficient margin. The problem is key exchange. Mechanisms based on asymmetric cryptography, such as Diffie-Hellman or RSA, are vulnerable to Shor's algorithm. It is they, not AES itself, that constitute the weak point from the perspective of quantum cryptography. This concerns both IKEv2 in IPsec and the MKA protocol in MACsec.

AES-256 itself is not the problem. Grover's algorithm halves the effective key strength, which at 256 bits still gives a level corresponding to 128 bits, that is a sufficient margin. The weak point is key exchange based on asymmetric cryptography, vulnerable to Shor's algorithm.
Furthest
IPsec

Has the most advanced standardisation work on hybrid key exchange, combining a classical mechanism with a post-quantum one. Commercial solutions are already available.

In progress
MACsec

The subject of active research into post-quantum key agreement. Publications show working prototypes, but production availability remains limited.

A different path
Layer 1, OTNsec

Key exchange via the auxiliary channel in the frame opens the way to feeding a key from an external source, which can be QKD. Interception of the key is detectable from the laws of physics, not from computational assumptions.

The harvest-now-decrypt-later scenario is why the matter is urgent already today. An adversary can intercept and archive encrypted traffic now, to decrypt it in ten or fifteen years. For data with a long sensitivity window, such as medical records, personal data, or trade secrets, the risk is real now. Layer one has a special property here, because the key exchange takes place via an auxiliary channel in the frame, which opens the way to feeding it a key from an external source. That source can be quantum key distribution, that is QKD. The key is then generated by a separate device using a quantum stream, and its interception by an attacker is detectable from the laws of physics, not from computational assumptions.

1
Inventory

Determine which routes carry data with a sensitivity window longer than ten years, because those are the target of the harvest-now-decrypt-later scenario.

2
Protect the routes with zero-overhead encryption

Cover those routes with layer one encryption, which costs neither throughput nor latency, and also hides metadata.

3
An external key source for the most critical routes

Introduce a key resistant to quantum attack, for example from QKD, fed via the auxiliary channel in the frame.

How to test before deployment


Manufacturers' declarations about latency and throughput are measured in laboratory conditions. In your network the results may be different, so a pilot on a single link before a decision on scale is an investment that pays back with the first problem detected.

Before enabling encryption
01Round-trip latency on the target route, under a load close to production, not on an empty link
02Effective throughput for the packet sizes of your traffic, because with small packets the overhead grows as a percentage
03Packet-size distribution in production traffic, because it decides how the test translates into a real loss
04Optical path parameters, received power and OSNR, as a reference point for diagnostics
the differencethe real cost of encryption in your network
After enabling encryption
01Round-trip latency
02Effective throughput
03Packet-size distribution
04Optical path parameters
The same four items, under the same conditions

The pilot is worth running for at least two weeks, to cover a full weekly traffic cycle including the weekend and periods of increased load.

Loss of key synchronisation

What happens to the traffic when the key agreement process is interrupted. Whether the link rebuilds itself and after how long.

Switchover to the backup route

Whether encryption is preserved after the switchover and how long the session takes to rebuild.

Behaviour after a device restart

Whether the configuration and key material survive a restart, or whether intervention is required.

Visibility in the management system

Whether the state of the encrypted session is reported to monitoring and whether a failure generates an alarm, or only a silent degradation.

The rollback path

How quickly encryption can be disabled if the pilot turns out badly, and whether it requires a maintenance window.

Duration: run the pilot for at least two weeks, to cover a full weekly traffic cycle including the weekend and periods of increased load.

A summary in three sentences


If you are protecting a route between two locations that you control, and you care about throughput or latency, choose layer one. If you are building an Ethernet network with many nodes on a limited budget, choose MACsec. If the traffic passes through infrastructure you do not control, you have to use IPsec, regardless of the performance cost. In most real deployments these layers complement rather than exclude one another. Layer one encryption on the backbone routes plus IPsec for selected applications is the arrangement we see most often with customers that have regulatory requirements.

Sources:

‍Microchip, In-flight Encryption in Service Provider Networks, a comparison of OTN, MACsec, and IPsec overhead, Cho, Sergeev, Post-quantum MACsec in Ethernet Networks, 32-byte overhead, 1 to 3 µs latency, throughput measurements, Automatic Intent-Based Secure Service Creation, arXiv, properties of physical-layer encryption, MACsec and IPsec, Chaotic Encryption for 10-Gb Ethernet Optical Links, arXiv, the effect of IPsec on throughput, Multi-Layer Encryption in Optical Fiber Networks, a comparison of the latencies and scope of protection of the three layers

FAQ:

FAQ, encryption in the network layers

The choice does not depend on the level of security, because all three layers encrypt with the same AES-256, but on which route you are protecting and where the trust boundary runs. In short: if you are protecting a route between two locations that you control, and you care about throughput or latency, choose layer one, that is optical encryption. If you are building an Ethernet network with many nodes on a limited budget, choose MACsec. If the traffic passes through infrastructure you do not control, you have to use IPsec, regardless of the performance cost. In most real deployments these layers complement rather than exclude one another, for example layer one on the backbone plus IPsec for selected applications.
The difference does not stem from the strength of the cipher, because that is the same everywhere, usually AES-256, but from how large a portion of the communication a given layer covers and over what section. The key factor is the scope of metadata protection, which in risk analysis often weighs more than the confidentiality of the content itself. Layer one encryption also hides traffic patterns, so an observer tapping the fibre does not know who is communicating with whom or when the intensity rises. MACsec leaves the MAC addresses visible, and IPsec the outer IP addresses, and from both the topology and activity of the network can be reconstructed. The second important dimension is the reach of protection, that is where the data exists in the clear.
Because MACsec protects a single link section, that is a hop, not the whole route. When data passes through several layer-two devices, each of them decrypts the frame and encrypts it again, so inside every device along the way the data exists in the clear. This has a direct consequence for the threat model: if it includes the intermediate devices, for example switches over which you do not have full control, MACsec is not the answer. In that situation protection comes either from IPsec, which protects end to end and which intermediate devices cannot decrypt, or from layer one on the specific section. The MAC headers with MACsec remain visible in any case, because they are needed for frame switching.
Layer one encryption, that is OTNsec, operates at the level of the OTN frame, before the signal is sent into the fibre. The payload is encrypted one to one, with no bytes added, and the authentication data fits into reserved fields of the OTN header. Hence zero overhead and full throughput. Everything that passes through the path is encrypted, regardless of protocol, so to an observer tapping the fibre neither addresses nor traffic patterns are visible, and that is precisely what industry materials call black fibre. The price is clear: the solution requires dedicated hardware on both sides of the link, works point to point, and cannot be stretched across a route with another vendor's device in the middle.
AES-256 itself is not the problem and is regarded as resistant to known quantum algorithms. Grover's algorithm halves the effective key strength, which at 256 bits still leaves a security level corresponding to 128 bits, and that is a sufficient margin. The weak point is key exchange, because mechanisms based on asymmetric cryptography, such as Diffie-Hellman or RSA, are vulnerable to Shor's algorithm. This concerns both the IKEv2 protocol in IPsec and the MKA protocol in MACsec. It is not the symmetric cipher itself, but precisely the key agreement that has to be secured from the perspective of quantum cryptography.
It is the scenario in which an adversary already today intercepts and archives encrypted traffic, in order to decrypt it in ten or fifteen years, once quantum computers capable of breaking today's key exchange mature. For data with a long sensitivity window, such as medical records, personal data, or trade secrets, the risk is therefore real already now, even though the breaking itself will happen in the future. The recommended migration path has three steps: first an inventory of which routes carry data with a sensitivity window longer than ten years, then protecting those routes with zero-overhead encryption in layer one, and finally, for the most critical routes, introducing an external source of a key resistant to quantum attack, for example from QKD.
Manufacturers' declarations are measured in the laboratory, so in your network the result tends to be different. Before enabling, measure four things: round-trip latency under a load close to production, effective throughput for the packet sizes of your traffic, the packet-size distribution, and the optical path parameters, that is received power and OSNR. After enabling, measure the same four items under the same conditions, and the difference is the real cost of encryption. In the pilot do not skip the behaviour on loss of key synchronisation, the switchover to the backup route, the behaviour after a device restart, the visibility of the session in the management system, and the rollback path. Run the pilot for at least two weeks, to cover a full weekly cycle including the weekend.
Yes, and in audited scenarios it has an additional advantage. Requirements for protecting data in transit are usually met by AES-256 encryption regardless of the layer, but auditors also ask whether the data exists anywhere in the clear along the route. With layer one encryption the answer is no, without reservation, because the data is not decrypted anywhere between the endpoints. The data-protection regulations and the national cybersecurity system speak of adequate technical measures, without indicating a specific layer, so MACsec too, deployed consistently on all links, meets these requirements, provided a risk analysis is documented that justifies the choice. The key, therefore, is not only the encryption itself but also supporting the choice of layer with a sound risk analysis.
Rozpocznij z nami nowy projekt!
Dziękujemy! Otrzymaliśmy Twoje zgłoszenie!
Ups! Wystąpił błąd podczas wysyłania formularza.

How to Deploy a Coherent QSFP-DD Module in an Existing DWDM Network?

You have a working DWDM network. You want to plug a coherent QSFP-DD module straight into the router and remove the transponder from the architecture. It sounds simple and it really is — but only if you know about a few things that can block the project before you even order the hardware. This is a practical guide. Not theory, just the order of operations, a checklist, and the mistakes engineers regret after the fact.
Czytaj więcej

Your fiber can carry 40 times more data. You probably haven't been told that.

CWDM, DWDM, and O-Band on a single fiber. Why now is the right time and how to get more out of the infrastructure you already have. Without laying new cables.
Czytaj więcej

QSFP28 DWDM - Scalability Simplicity for DCI

Can we find a more economical solution for connecting two or more data centers over short and medium distances?
Czytaj więcej