ResNet — The Shortcut That Made 152 Layers Trainable

ResNet (Residual Network) is a deep neural network proposed in 2015 by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun of Microsoft Research. Their paper, "Deep Residual Learning for Image Recognition", was posted to arXiv on 10 December 2015, the same day Microsoft announced that its team had taken first place in all three categories it entered at the ILSVRC 2015 image-recognition challenge: classification, localization and detection. The key idea is a shortcut that adds a block's input straight back to its output every few layers, which made it possible to train networks 152 layers deep, very deep for the time.

Put simply, ResNet showed how to get past a wall: networks should improve as they get deeper, yet in practice they got worse. The idea spread well beyond image recognition, into later translation models and the Transformer.

Deeper got worse: the degradation problem

Since AlexNet in 2012, the best ImageNet results had come from deeper networks. The paper notes that the leading methods used "very deep" models of sixteen to thirty layers. So would simply stacking more layers do even better?

The paper's answer was no. As layers were added, accuracy first saturated and then degraded rapidly, and the degradation was not caused by overfitting: the error on the training data itself went up. In the paper's Figure 1, a plain 56-layer network on CIFAR-10 has higher training and test error than a 20-layer one. The authors called this the degradation problem and took it as a sign that deeper networks are harder to optimize.

In principle a deeper model should never do worse than a shallower one, because you can build it from the shallow model plus extra layers that do nothing (identity mappings). Yet in actual training, solvers could not even find that solution.

Residual learning: learn only the difference

He and colleagues therefore had each group of layers learn not the target function itself but the difference from its input, the residual. A block's output becomes "what the layers computed plus the input itself", and the path that carries the input around the layers and adds it back is called a shortcut connection. If an identity mapping is what is needed, the layers only have to push their output towards zero, which the authors reasoned is much easier than learning to do nothing from scratch.

Identity shortcuts add neither extra parameters nor computation, and with them training kept working as the number of layers grew greatly.

ILSVRC 2015 and COCO 2015: first in five categories

According to the official ILSVRC results, the MSRA (Microsoft Research Asia) team scored as follows in the categories using only the provided training data.

The margins in localization and detection were wide, but in classification the gap to the next entry was just 0.014 percentage points. On classification alone ResNet's win was narrow; the work mattered less for its ranking than for showing a general way past the depth barrier. The team also took first place in COCO 2015 detection and segmentation.

On the day of the announcement, Microsoft's AI blog quoted Jian Sun, who led the project at the company's Beijing lab: "We even didn't believe this single idea could be so significant." Peter Lee, a corporate vice president at Microsoft Research, said the result destroyed some of the assumptions he had been making about how deep neural networks work. The paper went on to win the Best Paper Award at CVPR 2016 in June 2016.

Who gained what

The same blog post said the advances were already being used to improve Microsoft Project Oxford, a set of developer tools for speech and image recognition. The company claimed the system's accuracy meets and sometimes exceeds human-level performance. But in the same article the researchers themselves cautioned that this did not mean computers were getting smarter than humans in a general way, only that they were getting very good at narrow tasks such as identifying images in a database.

The clearest effect was on research. Google's neural machine translation system, announced in September 2016, added residual connections between its stacked LSTM layers so that it could use eight of them, citing the ResNet paper. The 2017 Transformer put a residual connection around every sub-layer of its encoder and decoder, again citing ResNet. Most of today's large language models descend from that Transformer design.

Citations tell the same story. According to a profile by the Chinese University of Hong Kong, a 2025 Nature analysis ranked the paper as the most-cited paper published in the twenty-first century.

For the competition's background see the ImageNet dataset and the AlexNet page; for the wider story, the AI timeline. The research division's history is on the Microsoft page.