Infrared and visible image fusion using two-layer generative adversarial network

Authors: Chen, L., Han, J. and Tian, F.

Journal: Journal of Intelligent and Fuzzy Systems

Volume: 40

Issue: 6

Pages: 11897-11913

eISSN: 1875-8967

ISSN: 1064-1246

DOI: 10.3233/JIFS-210041

Abstract:

Infrared (IR) images can distinguish targets from their backgrounds based on difference in thermal radiation, whereas visible images can provide texture details with high spatial resolution. The fusion of the IR and visible images has many advantages and can be applied to applications such as target detection and recognition. This paper proposes a two-layer generative adversarial network (GAN) to fuse these two types of images. In the first layer, the network generate fused images using two GANs: one uses the IR image as input and the visible image as ground truth, and the other with the visible as input and the IR as ground truth. In the second layer, the network transfer one of the two fused images generated in the first layer as input and the other as ground truth to GAN to generate the final fused image. We adopt TNO and INO data sets to verify our method, and by comparing with eight objective evaluation parameters obtained by other ten methods. It is demonstrated that our method is able to achieve better performance than state-of-arts on preserving both texture details and thermal information.

http://eprints.bournemouth.ac.uk/35907/

Source: Scopus

Infrared and visible image fusion using two-layer generative adversarial network

Authors: Chen, L., Han, J. and Tian, F.

Journal: JOURNAL OF INTELLIGENT & FUZZY SYSTEMS

Volume: 40

Issue: 6

Pages: 11897-11913

eISSN: 1875-8967

ISSN: 1064-1246

DOI: 10.3233/JIFS-210041

http://eprints.bournemouth.ac.uk/35907/

Source: Web of Science (Lite)

Infrared and visible image fusion using two-layer generative adversarial network

Authors: Chen, L., Han, J. and Tian, F.

Journal: Journal of Intelligent and Fuzzy Systems

Volume: 40

Issue: 6

Pages: 11897-11913

ISSN: 1064-1246

Abstract:

Infrared (IR) images can distinguish targets from their backgrounds based on difference in thermal radiation, whereas visible images can provide texture details with high spatial resolution. The fusion of the IR and visible images has many advantages and can be applied to applications such as target detection and recognition. This paper proposes a two-layer generative adversarial network (GAN) to fuse these two types of images. In the first layer, the network generate fused images using two GANs: one uses the IR image as input and the visible image as ground truth, and the other with the visible as input and the IR as ground truth. In the second layer, the network transfer one of the two fused images generated in the first layer as input and the other as ground truth to GAN to generate the final fused image. We adopt TNO and INO data sets to verify our method, and by comparing with eight objective evaluation parameters obtained by other ten methods. It is demonstrated that our method is able to achieve better performance than state-of-arts on preserving both texture details and thermal information.

http://eprints.bournemouth.ac.uk/35907/

https://content.iospress.com/articles/journal-of-intelligent-and-fuzzy-systems/ifs210041

Source: BURO EPrints