Layout Generation: Automated Components Placement for Advertising Poster using Transformer-Based from Layout Graph

Aisyah Dliya Ramadhanti; Kemas Rahmat Saleh Wiharja; Azmi Nurzakiah; Yoga Yustiawan

doi:10.22219/kinetik.v9i4.2035

Issue

Vol. 9, No. 4, November 2024 (Article in Progress)

Issue Published : Aug 19, 2024

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Layout Generation: Automated Components Placement for Advertising Poster using Transformer-Based from Layout Graph

https://doi.org/10.22219/kinetik.v9i4.2035

Aisyah Dliya Ramadhanti

Telkom University

Kemas Rahmat Saleh Wiharja

Telkom University

Azmi Nurzakiah

Digital BRIBRAIN

Yoga Yustiawan

Digital BRIBRAIN

Corresponding Author(s) : Kemas Rahmat Saleh Wiharja

bagindokemas@telkomuniversity.ac.id

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 9, No. 4, November 2024 (Article in Progress)
Article Published : Aug 19, 2024

Abstract

In the digital era, graphic design plays an important role in a company's marketing strategy, especially advertising posters that can convey messages to the audience. However, the process of making attractive and informative posters takes a long time, especially in the placement of components in the layout. This research aims to develop a layout generator system that focuses on placing components on the layout using the SGTransformer method. SGTransformer is one of the transformer-based methods that can be used to generate layouts. This research utilizes the capabilities of Laplacian positional encoding (LapPE) and edge features in SGTransformer that can improve the model's performance in developing a more structured layout based on the input layout graph. A layout graph describes the spatial relationship between components in a layout. The SGTransformer model is trained using advertising poster datasets collected from social media. Once trained, the model automatically places components based on the description of the spatial relationship between components represented by the layout graph. Evaluation of the model shows that the SGTransformer method can produce structured and more diverse layouts although there are still challenges such as overlap between components. This research contributes to the efficiency of the advertising poster design process and provides practical solutions to the creation of advertising posters in the digital marketing industry. Code and other materials will be released at https://github.com/syahdeee/Layout-Generator.

Keywords

Layout Generation, Transformer-Based Model, Layout Graph, Advertising Poster, Automated Components Positioning

Ramadhanti, A. D., Wiharja, K. R. S., Nurzakiah, A., & Yustiawan, Y. (2024). Layout Generation: Automated Components Placement for Advertising Poster using Transformer-Based from Layout Graph. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 9(4). https://doi.org/10.22219/kinetik.v9i4.2035

Download Citation

References

E. Setiawan Nababan, “IMPLEMENTATION OF ADVERTISING POSTER AS A PROMOTIONAL MEDIA FOR MSME.”
K. J. Murchie and D. Diomede, “Fundamentals of Graphic Design-essential tools for effective visual science communication,” Facets, vol. 5, no. 1. Canadian Science Publishing, pp. 409–422, Jun. 11, 2020. doi: 10.1139/FACETS-2018-0049.
J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators,” Jan. 2019, doi: https://doi.org/10.48550/arXiv.1901.06767.
A. A. Jyothi, T. Durand, J. He, L. Sigal, and G. Mori, “LayoutVAE: Stochastic Scene Layout Generation From a Label Set,” Jul. 2019, doi: https://doi.org/10.48550/arXiv.1907.10719.
X. Zheng, X. Qiao, Y. Cao, and R. W. H. Lau, “Content-aware generative modeling of graphic design layouts,” ACM Trans Graph, vol. 38, no. 4, Jul. 2019, doi: 10.1145/3306346.3322971.
J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Synthesizing Graphic Layouts with Vector-Wireframe Adversarial Networks,” IEEE Trans Pattern Anal Mach Intell, vol. 43, no. 7, pp. 2388–2399, Jul. 2021, doi: 10.1109/TPAMI.2019.2963663.
H.-Y. Lee et al., “Neural Design Network: Graphic Layout Generation with Constraints,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09421.
S. Chai, L. Zhuang, and F. Yan, “LayoutDM: Transformer-based Diffusion Model for Layout Generation,” May 2023, doi: https://doi.org/10.48550/arXiv.2305.02567.
D. M. Arroyo, J. Postels, and F. Tombari, “Variational Transformer Networks for Layout Generation.” doi: https://doi.org/10.48550/arXiv.2104.02416.
J. Johnson, A. Gupta, and L. Fei-Fei, “Image Generation from Scene Graphs,” Apr. 2018, doi: https://doi.org/10.48550/arXiv.1804.01622.
R. Sortino, S. Palazzo, and C. Spampinato, “Transformer-based Image Generation from Scene Graphs,” Mar. 2023, doi: https://doi.org/10.48550/arXiv.2303.04634.
E. Quiring, A. Müller, and K. Rieck, “On the Detection of Image-Scaling Attacks in Machine Learning,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Dec. 2023, pp. 506–520. doi: 10.1145/3627106.3627134.
“Automated Catalog Generation using Deep Learning,” International Research Journal of Modernization in Engineering Technology and Science, Aug. 2023, doi: 10.56726/irjmets44010.
P. S. Parsania and P. V Virparia, “International Journal on Recent and Innovation Trends in Computing and Communication Performance Analysis of Image Scaling Algorithms”, doi: https://doi.org/10.17762/ijritcc.v4i6.2359.
O. I. Khalaf, C. A. T. Romero, A. Azhagu Jaisudhan Pazhani, and G. Vinuja, “VLSI Implementation of a High-Performance Nonlinear Image Scaling Algorithm,” J Healthc Eng, vol. 2021, 2021, doi: 10.1155/2021/6297856.
G. Zhu et al., “Scene Graph Generation: A Comprehensive Survey,” Jan. 2022, doi: https://doi.org/10.48550/arXiv.2201.00443.
S. Khandelwal and L. Sigal, “Iterative Scene Graph Generation,” Jul. 2022, doi: https://doi.org/10.48550/arXiv.2207.13440.
M. Wang et al., “Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks,” Sep. 2019, doi: https://doi.org/10.48550/arXiv.1909.01315
V. P. Dwivedi and X. Bresson, “A Generalization of Transformer Networks to Graphs,” Dec. 2020, doi: https://doi.org/10.48550/arXiv.2012.09699.
Z. Chen et al., “PIoU Loss: Towards Accurate Oriented Object Detection in Complex Environments,” Jul. 2020, doi: https://doi.org/10.48550/arXiv.2007.09584.
Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.08287.
I. Ullah, M. Manzo, M. Shah, and M. Madden, “Graph Convolutional Networks: analysis, improvements and results,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09592.
A. Sobolevsky, G.-A. Bilodeau, J. Cheng, and J. L. C. Guo, “GUILGET: GUI Layout GEneration with Transformer,” Apr. 2023, doi: https://doi.org/10.48550/arXiv.2304.09012.
J. Li, J. Yang, J. Zhang, C. Liu, C. Wang, and T. Xu, “Attribute-conditioned Layout GAN for Automatic Graphic Design,” Sep. 2020, doi: https://doi.org/10.48550/arXiv.2009.05284.
R. Carletto, H. Cardot, and N. Ragot, “Deep Learning for Document Layout Generation: A First Reproducible Quantitative Evaluation and a Baseline Model,” pp. 20–35, 2021, doi: https://doi.org/10.1007/978-3-030-86334-0_2.
Q. Jing et al., “Layout Generation for Various Scenarios in Mobile Shopping Applications,” in Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, Apr. 2023. doi: 10.1145/3544548.3581446.
K. Kikuchi, E. Simo-Serra, M. Otani, and K. Yamaguchi, “Constrained Graphic Layout Generation via Latent Optimization,” in MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, Oct. 2021, pp. 88–96. doi: 10.1145/3474085.3475497.
E. Min et al., “Transformer for Graphs: An Overview from Architecture Perspective,” Feb. 2022, doi: https://doi.org/10.48550/arXiv.2202.08455.
S. Yun, M. Jeong, R. Kim, J. Kang, and H. J. Kim, “Graph Transformer Networks,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.06455.

References

E. Setiawan Nababan, “IMPLEMENTATION OF ADVERTISING POSTER AS A PROMOTIONAL MEDIA FOR MSME.”

K. J. Murchie and D. Diomede, “Fundamentals of Graphic Design-essential tools for effective visual science communication,” Facets, vol. 5, no. 1. Canadian Science Publishing, pp. 409–422, Jun. 11, 2020. doi: 10.1139/FACETS-2018-0049.

J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators,” Jan. 2019, doi: https://doi.org/10.48550/arXiv.1901.06767.

A. A. Jyothi, T. Durand, J. He, L. Sigal, and G. Mori, “LayoutVAE: Stochastic Scene Layout Generation From a Label Set,” Jul. 2019, doi: https://doi.org/10.48550/arXiv.1907.10719.

X. Zheng, X. Qiao, Y. Cao, and R. W. H. Lau, “Content-aware generative modeling of graphic design layouts,” ACM Trans Graph, vol. 38, no. 4, Jul. 2019, doi: 10.1145/3306346.3322971.

J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Synthesizing Graphic Layouts with Vector-Wireframe Adversarial Networks,” IEEE Trans Pattern Anal Mach Intell, vol. 43, no. 7, pp. 2388–2399, Jul. 2021, doi: 10.1109/TPAMI.2019.2963663.

H.-Y. Lee et al., “Neural Design Network: Graphic Layout Generation with Constraints,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09421.

S. Chai, L. Zhuang, and F. Yan, “LayoutDM: Transformer-based Diffusion Model for Layout Generation,” May 2023, doi: https://doi.org/10.48550/arXiv.2305.02567.

D. M. Arroyo, J. Postels, and F. Tombari, “Variational Transformer Networks for Layout Generation.” doi: https://doi.org/10.48550/arXiv.2104.02416.

J. Johnson, A. Gupta, and L. Fei-Fei, “Image Generation from Scene Graphs,” Apr. 2018, doi: https://doi.org/10.48550/arXiv.1804.01622.

R. Sortino, S. Palazzo, and C. Spampinato, “Transformer-based Image Generation from Scene Graphs,” Mar. 2023, doi: https://doi.org/10.48550/arXiv.2303.04634.

E. Quiring, A. Müller, and K. Rieck, “On the Detection of Image-Scaling Attacks in Machine Learning,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Dec. 2023, pp. 506–520. doi: 10.1145/3627106.3627134.

“Automated Catalog Generation using Deep Learning,” International Research Journal of Modernization in Engineering Technology and Science, Aug. 2023, doi: 10.56726/irjmets44010.

P. S. Parsania and P. V Virparia, “International Journal on Recent and Innovation Trends in Computing and Communication Performance Analysis of Image Scaling Algorithms”, doi: https://doi.org/10.17762/ijritcc.v4i6.2359.

O. I. Khalaf, C. A. T. Romero, A. Azhagu Jaisudhan Pazhani, and G. Vinuja, “VLSI Implementation of a High-Performance Nonlinear Image Scaling Algorithm,” J Healthc Eng, vol. 2021, 2021, doi: 10.1155/2021/6297856.

G. Zhu et al., “Scene Graph Generation: A Comprehensive Survey,” Jan. 2022, doi: https://doi.org/10.48550/arXiv.2201.00443.

S. Khandelwal and L. Sigal, “Iterative Scene Graph Generation,” Jul. 2022, doi: https://doi.org/10.48550/arXiv.2207.13440.

M. Wang et al., “Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks,” Sep. 2019, doi: https://doi.org/10.48550/arXiv.1909.01315

V. P. Dwivedi and X. Bresson, “A Generalization of Transformer Networks to Graphs,” Dec. 2020, doi: https://doi.org/10.48550/arXiv.2012.09699.

Z. Chen et al., “PIoU Loss: Towards Accurate Oriented Object Detection in Complex Environments,” Jul. 2020, doi: https://doi.org/10.48550/arXiv.2007.09584.

Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.08287.

I. Ullah, M. Manzo, M. Shah, and M. Madden, “Graph Convolutional Networks: analysis, improvements and results,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09592.

A. Sobolevsky, G.-A. Bilodeau, J. Cheng, and J. L. C. Guo, “GUILGET: GUI Layout GEneration with Transformer,” Apr. 2023, doi: https://doi.org/10.48550/arXiv.2304.09012.

J. Li, J. Yang, J. Zhang, C. Liu, C. Wang, and T. Xu, “Attribute-conditioned Layout GAN for Automatic Graphic Design,” Sep. 2020, doi: https://doi.org/10.48550/arXiv.2009.05284.

R. Carletto, H. Cardot, and N. Ragot, “Deep Learning for Document Layout Generation: A First Reproducible Quantitative Evaluation and a Baseline Model,” pp. 20–35, 2021, doi: https://doi.org/10.1007/978-3-030-86334-0_2.

Q. Jing et al., “Layout Generation for Various Scenarios in Mobile Shopping Applications,” in Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, Apr. 2023. doi: 10.1145/3544548.3581446.

K. Kikuchi, E. Simo-Serra, M. Otani, and K. Yamaguchi, “Constrained Graphic Layout Generation via Latent Optimization,” in MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, Oct. 2021, pp. 88–96. doi: 10.1145/3474085.3475497.

E. Min et al., “Transformer for Graphs: An Overview from Architecture Perspective,” Feb. 2022, doi: https://doi.org/10.48550/arXiv.2202.08455.

S. Yun, M. Jeong, R. Kim, J. Kang, and H. J. Kim, “Graph Transformer Networks,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.06455.

Author Biographies

Azmi Nurzakiah, Digital BRIBRAIN

Digital Banking Development and Operation Division, Bank Rakyat Indonesia (BRI)

Yoga Yustiawan, Digital BRIBRAIN

Digital Banking Development and Operation Division, Bank Rakyat Indonesia (BRI)

Issue

Vol. 9, No. 4, November 2024 (Article in Progress)

Layout Generation: Automated Components Placement for Advertising Poster using Transformer-Based from Layout Graph

Corresponding Author(s) : Kemas Rahmat Saleh Wiharja

Abstract

Keywords

Download Citation

References

Author Biographies

Downloads