An Extensive Survey on Sketch to Photo Synthesis Methods: Trends and Challenges
Keywords:
Deep learning, Face sketch, Model driven, Data-drivenAbstract
Nowadays Face sketch- photo synthesis has drawn the attention of many researchers. Since the Photo to sketch synthesis or sketch to photo synthesis is used in many different application, detailed review is very much important to analyse state-of-the-art approaches. With this in mind, we offer a thorough analysis of the existing deep learning-based and traditional approaches, which fall into the categories of data-driven and model-driven approaches, in this study. A comparative study of the evaluated methods is conducted by considering several factors like the performance measurements, algorithms, and dataset.
Downloads
References
Q. Liu, X. Tang, H. Jin, H. Lu, and S. Ma, “A nonlinear approach for face sketch synthesis and recognition,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., San Diego, CA, USA, Jun. 2005, pp. 1005–1010
H. Zhou, Z. Kuang, and K.-Y. K. Wong, “Markov weight fields for face sketch synthesis,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 1091–1097.
N. Wang, D. Tao, X. Gao, X. Li, and J. Li, “Transductive face sketchphoto synthesis,” IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 9, pp. 1364–1376, Sep. 2013
R. G. Uhl, Jr., N. da Vitoria Lobo, and Y. H. Kwon, “Recognizing a facial image from a police sketch,” in Proc. 2nd IEEE Workshop Appl. Comput. Vis., Sarasota, FL, USA, Dec. 1994, pp. 129–137
R. G. Uhl, Jr., and N. da Vitoria Lobo, “A framework for recognizing a facial image from a police sketch,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., San Francisco, CA, USA, Jun. 1996, pp. 586–593.
X. Tang and X. Wang, “Face sketch recognition,” IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 1, pp. 50–57, Jan. 2004
X. Tang and X. Wang, “Face sketch synthesis and recognition,” in Proc. 9th IEEE Int. Conf. Comput. Vis., Nice, France, Oct. 2003, pp. 687–694
H. Koshimizu, M. Tominaga, T. Fujiwara, K. Murakami. On KANSEI facial image processing for computerized facial caricaturing system PICASSO. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics,IEEE, Tokyo, Japan, pp.294–299, 1999. DOI: 10.1109/ICSMC.1999.816567.
J. Yu, M. Wang, and D. Tao, “Semisupervised multiview distance metric learning for cartoon synthesis,” IEEE Trans. Image Process., vol. 21, no. 11, pp. 4636–4648, Nov. 2012.
X. Gao, N. Wang, D. Tao, and X. Li, “Face sketchphoto synthesis and retrieval using sparse representation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 8, pp. 1213–1226, 2012.
Y. Song, L. Bao, Q. Yang, and M. H. Yang, “Real-time exemplar-based face sketch synthesis,” in European Conference on Computer Vision, 2014, pp. 800–813.
Q. Pan, Y. Liang, L. Zhang, and S. Wang, “Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis,” in Computer Vision and Pattern Recognition, 2012, pp. 2216– 2223.
S. Zhang, X. Gao, N. Wang, J. Li, and M. Zhang, “Face sketch synthesis via sparse representation-based greedy search.” IEEE Transactions on Image Processing, vol. 24, no. 8, pp. 2466–77, 2015.
S. Zhang, X. Gao, N. Wang, and J. Li, “Robust face sketch style synthesis,” IEEE Transactions on Image Processing, vol. 25, no. 1, p. 220, 2016.
N. Wang, D. Tao, X. Gao, X. Li, and J. Li, “Transductive face sketchphoto synthesis,” IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 9, pp. 1364–1376, 2013.
Gelman, A., Carlin, H., Stern, J., & Rubin, D. (2003). Bayesian data analysis. Boca Raton: Chapman & Hall/CRC
X. Wang and X. Tang, “Face photo-sketch synthesis and recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 11, pp. 1955–1967, Nov. 2009.
W. Zhang, X. Wang, and X. Tang, “Lighting and pose robust face sketch synthesis,” in Proc. 11th Eur. Conf. Comput. Vis., Heraklion, Greece, Sep. 2010, pp. 420–433
H. Zhou, Z. Kuang, and K.-Y. K. Wong, “Markov weight fields for face sketch synthesis,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 1091–1097.
X. Gao, J. Zhong, J. Li, and C. Tian, “Face sketch synthesis algorithm based on E-HMM and selective ensemble,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 4, pp. 487–496, Apr. 2008.
B. Xiao, X. Gao, D. Tao, and X. Li, “A new approach for face recognition by sketches in photos,” Signal Process., vol. 89, no. 8, pp. 1576–1588, 2009
I. T. Jolliffe, Principal Component Analysis. Hoboken, NJ, USA: Wiley, 2002
X. Tang and X. Wang, “Face photo recognition using sketch,” in Proc. IEEE Int. Conf. Image Process., Rochester, NY, USA, Sep. 2002, pp. I-257–I-260.
Liu, W., Tang, X., & Liu, J. (2007b). Bayesian tensor inference for sketch-based face photo hallucination. In Proceedings of international joint conference on artificial intelligence (pp. 2141– 2146).
M. Chang, L. Zhou, Y. Han, and X. Deng, “Face sketch synthesis via sparse representation,” in Proc. 20th Int. Conf. Pattern Recognit., Istanbul, Turkey, Aug. 2010, pp. 2146–2149.
H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proc. 21st Annu. Conf. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2007, pp. 801–808
X. Gao, N. Wang, D. Tao, and X. Li, “Face sketch–photo synthesis and retrieval using sparse representation,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 8, pp. 1213–1226, Aug. 2012.
J. Zhang, N. Wang, X. Gao, D. Tao, and X. Li, “Face sketch-photo synthesis based on support vector regression,” in Proc. 18th IEEE Int. Conf. Image Process., Brussels, Belgium, Sep. 2011, pp. 1125–1128.
N. Wang, X. Gao, D. Tao, and X. Li, “Face sketch-photo synthesis under multi-dictionary sparse representation framework,” in Proc. 6th Int. Conf. Image Graph., Hefei, China, Aug. 2011, pp. 82–87.
L. Zhang, L. Lin, X.Wu, S. Ding, and L. Zhang. “End-to-end photo-sketch generation via fully convolutional representation learning.,” in Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pages 627–634. ACM, 2015.
R. Socher, M. Ganjoo, C. D. Manning, and A. Ng. “Zero-shot learning through cross-modal transfer,” in Advances in neural information processing systems, pages 935–943, 2013.
M. Iranmanesh, A. Dabouei, H. Kazemi, and N. M. Nasrabadi. “Deep cross polarimetric thermal-to-visible face recognition,” in Biometrics (ICB), 2018 International Conference on. IEEE, 2018.
C. Li and M. Wand, “Combining markov random fields and convolutional neural networks for image synthesis,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2479–2486, 2016.
L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks”, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2414–2423, 2016.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets”, in Advances in neural information processing systems pages 2672–2680, 2014.
Lu Y., Wu S., Tai YW., Tang CK, “Image Generation from Sketch Constraint Using Contextual GAN”, in: Ferrari V., Hebert M., Sminchisescu C., Weiss Y. (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol 11220. Springer, Cham.
J. Zhao, X. Xie, L. Wang, M. Cao and M. Zhang, "Generating Photographic Faces from the Sketch Guided by Attribute Using GAN," in IEEE Access, vol. 7, pp. 23844-23851, 2019
Uche Osahor, Hadi Kazemi, Ali Dabouei, Nasser Nasrabadi,(2020), “Quality Guided Sketch-to-Photo Image Synthesis”, In: Computer Vision And Pattern Recognition (CVPR).
Yu et al., "Toward Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs," in IEEE Transactions on Cybernetics, doi: 10.1109/TCYB.2020
Chao, Wentao et al. “High-Fidelity Face Sketch-To-Photo Synthesis Using Generative Adversarial Network,” 2019 IEEE International Conference on Image Processing (ICIP) (2019): 4699-4703
J. Zheng, W. Song, Y. Wu, R. Xu and F. Liu, (2019), "Feature Encoder Guided Generative Adversarial Network for Face Photo-Sketch Synthesis", IEEE Access, vol. 7, pp. 154971-154985
Wang, V. Sindagi and V. Patel, "High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks," 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi'an, 2018, pp. 83-90.
Hadi Kazemi et al. “Facial Attributes Guided Deep Sketch-to-Photo Synthesis”,2018 IEEE Winter Conference on Applications of Computer Vision Workshops20
Yuke Fang Yuke Fang “Identity-aware CycleGAN for face photo-sketch synthesis and recognition,” Pattern Recognition,Volume 102,2020,107249,ISSN 0031-3203.
Wang, V. Sindagi and V. Patel.,"High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks", 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi'an, 2018, pp. 83-90.
W. Zhang, X. Wang, and X. Tang, “Coupled information-theoretic encoding for face photo-sketch recognition,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2011, pp. 513– 520.
X. B. Gao, N. N. Wang, D. C. Tao, X. L. Li., “Face sketch photo synthesis and retrieval using sparse representation,” IEEE Transactions on Circuits and Systems for Video Technology, vol.22, no.8, pp.1213–1226, 2012. DOI: 10. 1109/TCSVT.2012.2198090.
H. S. Bhatt, S. Bharadwaj, R. Singh, M. Vatsa, “On matching sketches with digital face images.,” in Proceedings of the 4th IEEE International Conference on Biometrics: Theory, Applications and Systems, IEEE, Washington DC, USA, 2010. DOI: 10.1109/BTAS.2010.5634 507.
R. Yi, M. F. Xia, Y. J. Liu, Y. K. Lai, P. L. Rosin, “Line drawings for face portraits from photos using global and local structure based GANs”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.43, no.10, pp.3462–3475, 2021. DOI: 10.1109/TPAMI.2020.2987 931
Fan, Deng-Ping, et al. "Facial-sketch synthesis: a new challenge." Machine Intelligence Research 19.4 (2022): 257-287.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
