This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Layout Generation: Automated Components Placement for Advertising Poster using Transformer-Based from Layout Graph
Corresponding Author(s) : Kemas Rahmat Saleh Wiharja
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control,
Vol. 9, No. 4, November 2024 (Article in Progress)
Abstract
In the digital era, graphic design plays an important role in a company's marketing strategy, especially advertising posters that can convey messages to the audience. However, the process of making attractive and informative posters takes a long time, especially in the placement of components in the layout. This research aims to develop a layout generator system that focuses on placing components on the layout using the SGTransformer method. SGTransformer is one of the transformer-based methods that can be used to generate layouts. This research utilizes the capabilities of Laplacian positional encoding (LapPE) and edge features in SGTransformer that can improve the model's performance in developing a more structured layout based on the input layout graph. A layout graph describes the spatial relationship between components in a layout. The SGTransformer model is trained using advertising poster datasets collected from social media. Once trained, the model automatically places components based on the description of the spatial relationship between components represented by the layout graph. Evaluation of the model shows that the SGTransformer method can produce structured and more diverse layouts although there are still challenges such as overlap between components. This research contributes to the efficiency of the advertising poster design process and provides practical solutions to the creation of advertising posters in the digital marketing industry. Code and other materials will be released at https://github.com/syahdeee/Layout-Generator.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- E. Setiawan Nababan, “IMPLEMENTATION OF ADVERTISING POSTER AS A PROMOTIONAL MEDIA FOR MSME.”
- K. J. Murchie and D. Diomede, “Fundamentals of Graphic Design-essential tools for effective visual science communication,” Facets, vol. 5, no. 1. Canadian Science Publishing, pp. 409–422, Jun. 11, 2020. doi: 10.1139/FACETS-2018-0049.
- J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators,” Jan. 2019, doi: https://doi.org/10.48550/arXiv.1901.06767.
- A. A. Jyothi, T. Durand, J. He, L. Sigal, and G. Mori, “LayoutVAE: Stochastic Scene Layout Generation From a Label Set,” Jul. 2019, doi: https://doi.org/10.48550/arXiv.1907.10719.
- X. Zheng, X. Qiao, Y. Cao, and R. W. H. Lau, “Content-aware generative modeling of graphic design layouts,” ACM Trans Graph, vol. 38, no. 4, Jul. 2019, doi: 10.1145/3306346.3322971.
- J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Synthesizing Graphic Layouts with Vector-Wireframe Adversarial Networks,” IEEE Trans Pattern Anal Mach Intell, vol. 43, no. 7, pp. 2388–2399, Jul. 2021, doi: 10.1109/TPAMI.2019.2963663.
- H.-Y. Lee et al., “Neural Design Network: Graphic Layout Generation with Constraints,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09421.
- S. Chai, L. Zhuang, and F. Yan, “LayoutDM: Transformer-based Diffusion Model for Layout Generation,” May 2023, doi: https://doi.org/10.48550/arXiv.2305.02567.
- D. M. Arroyo, J. Postels, and F. Tombari, “Variational Transformer Networks for Layout Generation.” doi: https://doi.org/10.48550/arXiv.2104.02416.
- J. Johnson, A. Gupta, and L. Fei-Fei, “Image Generation from Scene Graphs,” Apr. 2018, doi: https://doi.org/10.48550/arXiv.1804.01622.
- R. Sortino, S. Palazzo, and C. Spampinato, “Transformer-based Image Generation from Scene Graphs,” Mar. 2023, doi: https://doi.org/10.48550/arXiv.2303.04634.
- E. Quiring, A. Müller, and K. Rieck, “On the Detection of Image-Scaling Attacks in Machine Learning,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Dec. 2023, pp. 506–520. doi: 10.1145/3627106.3627134.
- “Automated Catalog Generation using Deep Learning,” International Research Journal of Modernization in Engineering Technology and Science, Aug. 2023, doi: 10.56726/irjmets44010.
- P. S. Parsania and P. V Virparia, “International Journal on Recent and Innovation Trends in Computing and Communication Performance Analysis of Image Scaling Algorithms”, doi: https://doi.org/10.17762/ijritcc.v4i6.2359.
- O. I. Khalaf, C. A. T. Romero, A. Azhagu Jaisudhan Pazhani, and G. Vinuja, “VLSI Implementation of a High-Performance Nonlinear Image Scaling Algorithm,” J Healthc Eng, vol. 2021, 2021, doi: 10.1155/2021/6297856.
- G. Zhu et al., “Scene Graph Generation: A Comprehensive Survey,” Jan. 2022, doi: https://doi.org/10.48550/arXiv.2201.00443.
- S. Khandelwal and L. Sigal, “Iterative Scene Graph Generation,” Jul. 2022, doi: https://doi.org/10.48550/arXiv.2207.13440.
- M. Wang et al., “Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks,” Sep. 2019, doi: https://doi.org/10.48550/arXiv.1909.01315
- V. P. Dwivedi and X. Bresson, “A Generalization of Transformer Networks to Graphs,” Dec. 2020, doi: https://doi.org/10.48550/arXiv.2012.09699.
- Z. Chen et al., “PIoU Loss: Towards Accurate Oriented Object Detection in Complex Environments,” Jul. 2020, doi: https://doi.org/10.48550/arXiv.2007.09584.
- Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.08287.
- I. Ullah, M. Manzo, M. Shah, and M. Madden, “Graph Convolutional Networks: analysis, improvements and results,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09592.
- A. Sobolevsky, G.-A. Bilodeau, J. Cheng, and J. L. C. Guo, “GUILGET: GUI Layout GEneration with Transformer,” Apr. 2023, doi: https://doi.org/10.48550/arXiv.2304.09012.
- J. Li, J. Yang, J. Zhang, C. Liu, C. Wang, and T. Xu, “Attribute-conditioned Layout GAN for Automatic Graphic Design,” Sep. 2020, doi: https://doi.org/10.48550/arXiv.2009.05284.
- R. Carletto, H. Cardot, and N. Ragot, “Deep Learning for Document Layout Generation: A First Reproducible Quantitative Evaluation and a Baseline Model,” pp. 20–35, 2021, doi: https://doi.org/10.1007/978-3-030-86334-0_2.
- Q. Jing et al., “Layout Generation for Various Scenarios in Mobile Shopping Applications,” in Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, Apr. 2023. doi: 10.1145/3544548.3581446.
- K. Kikuchi, E. Simo-Serra, M. Otani, and K. Yamaguchi, “Constrained Graphic Layout Generation via Latent Optimization,” in MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, Oct. 2021, pp. 88–96. doi: 10.1145/3474085.3475497.
- E. Min et al., “Transformer for Graphs: An Overview from Architecture Perspective,” Feb. 2022, doi: https://doi.org/10.48550/arXiv.2202.08455.
- S. Yun, M. Jeong, R. Kim, J. Kang, and H. J. Kim, “Graph Transformer Networks,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.06455.
References
E. Setiawan Nababan, “IMPLEMENTATION OF ADVERTISING POSTER AS A PROMOTIONAL MEDIA FOR MSME.”
K. J. Murchie and D. Diomede, “Fundamentals of Graphic Design-essential tools for effective visual science communication,” Facets, vol. 5, no. 1. Canadian Science Publishing, pp. 409–422, Jun. 11, 2020. doi: 10.1139/FACETS-2018-0049.
J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators,” Jan. 2019, doi: https://doi.org/10.48550/arXiv.1901.06767.
A. A. Jyothi, T. Durand, J. He, L. Sigal, and G. Mori, “LayoutVAE: Stochastic Scene Layout Generation From a Label Set,” Jul. 2019, doi: https://doi.org/10.48550/arXiv.1907.10719.
X. Zheng, X. Qiao, Y. Cao, and R. W. H. Lau, “Content-aware generative modeling of graphic design layouts,” ACM Trans Graph, vol. 38, no. 4, Jul. 2019, doi: 10.1145/3306346.3322971.
J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “LayoutGAN: Synthesizing Graphic Layouts with Vector-Wireframe Adversarial Networks,” IEEE Trans Pattern Anal Mach Intell, vol. 43, no. 7, pp. 2388–2399, Jul. 2021, doi: 10.1109/TPAMI.2019.2963663.
H.-Y. Lee et al., “Neural Design Network: Graphic Layout Generation with Constraints,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09421.
S. Chai, L. Zhuang, and F. Yan, “LayoutDM: Transformer-based Diffusion Model for Layout Generation,” May 2023, doi: https://doi.org/10.48550/arXiv.2305.02567.
D. M. Arroyo, J. Postels, and F. Tombari, “Variational Transformer Networks for Layout Generation.” doi: https://doi.org/10.48550/arXiv.2104.02416.
J. Johnson, A. Gupta, and L. Fei-Fei, “Image Generation from Scene Graphs,” Apr. 2018, doi: https://doi.org/10.48550/arXiv.1804.01622.
R. Sortino, S. Palazzo, and C. Spampinato, “Transformer-based Image Generation from Scene Graphs,” Mar. 2023, doi: https://doi.org/10.48550/arXiv.2303.04634.
E. Quiring, A. Müller, and K. Rieck, “On the Detection of Image-Scaling Attacks in Machine Learning,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Dec. 2023, pp. 506–520. doi: 10.1145/3627106.3627134.
“Automated Catalog Generation using Deep Learning,” International Research Journal of Modernization in Engineering Technology and Science, Aug. 2023, doi: 10.56726/irjmets44010.
P. S. Parsania and P. V Virparia, “International Journal on Recent and Innovation Trends in Computing and Communication Performance Analysis of Image Scaling Algorithms”, doi: https://doi.org/10.17762/ijritcc.v4i6.2359.
O. I. Khalaf, C. A. T. Romero, A. Azhagu Jaisudhan Pazhani, and G. Vinuja, “VLSI Implementation of a High-Performance Nonlinear Image Scaling Algorithm,” J Healthc Eng, vol. 2021, 2021, doi: 10.1155/2021/6297856.
G. Zhu et al., “Scene Graph Generation: A Comprehensive Survey,” Jan. 2022, doi: https://doi.org/10.48550/arXiv.2201.00443.
S. Khandelwal and L. Sigal, “Iterative Scene Graph Generation,” Jul. 2022, doi: https://doi.org/10.48550/arXiv.2207.13440.
M. Wang et al., “Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks,” Sep. 2019, doi: https://doi.org/10.48550/arXiv.1909.01315
V. P. Dwivedi and X. Bresson, “A Generalization of Transformer Networks to Graphs,” Dec. 2020, doi: https://doi.org/10.48550/arXiv.2012.09699.
Z. Chen et al., “PIoU Loss: Towards Accurate Oriented Object Detection in Complex Environments,” Jul. 2020, doi: https://doi.org/10.48550/arXiv.2007.09584.
Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.08287.
I. Ullah, M. Manzo, M. Shah, and M. Madden, “Graph Convolutional Networks: analysis, improvements and results,” Dec. 2019, doi: https://doi.org/10.48550/arXiv.1912.09592.
A. Sobolevsky, G.-A. Bilodeau, J. Cheng, and J. L. C. Guo, “GUILGET: GUI Layout GEneration with Transformer,” Apr. 2023, doi: https://doi.org/10.48550/arXiv.2304.09012.
J. Li, J. Yang, J. Zhang, C. Liu, C. Wang, and T. Xu, “Attribute-conditioned Layout GAN for Automatic Graphic Design,” Sep. 2020, doi: https://doi.org/10.48550/arXiv.2009.05284.
R. Carletto, H. Cardot, and N. Ragot, “Deep Learning for Document Layout Generation: A First Reproducible Quantitative Evaluation and a Baseline Model,” pp. 20–35, 2021, doi: https://doi.org/10.1007/978-3-030-86334-0_2.
Q. Jing et al., “Layout Generation for Various Scenarios in Mobile Shopping Applications,” in Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, Apr. 2023. doi: 10.1145/3544548.3581446.
K. Kikuchi, E. Simo-Serra, M. Otani, and K. Yamaguchi, “Constrained Graphic Layout Generation via Latent Optimization,” in MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, Oct. 2021, pp. 88–96. doi: 10.1145/3474085.3475497.
E. Min et al., “Transformer for Graphs: An Overview from Architecture Perspective,” Feb. 2022, doi: https://doi.org/10.48550/arXiv.2202.08455.
S. Yun, M. Jeong, R. Kim, J. Kang, and H. J. Kim, “Graph Transformer Networks,” Nov. 2019, doi: https://doi.org/10.48550/arXiv.1911.06455.