Transforming Hand Drawn Wireframes into Front-End Code with Deep Learning

作     者：Saman Riaz Ali Arshad Shahab S.Band Amir Mosavi 

作者机构：Department of Computer ScienceNational University of TechnologyIslamabad44000Pakistan Department of Computer ScienceInstitute of Space TechnologyIslamabad44000Pakistan Future Technology Research CenterNational Yunlin University of Science and TechnologyDouliu64002YunlinTaiwan Faculty of Civil EngineeringTechnische Universitat DresdenDresden 01069Germany 

出 版 物：《Computers, Materials & Continua》 (计算机、材料和连续体（英文）)

年 卷 期：2022年第72卷第9期

页      面：4303-4321页

核心收录：

学科分类：08[工学] 0805[工学-材料科学与工程（可授工学、理学学位）] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Sir Syed CASE Institute of Technology 

主　　题：Deep learning wireframes front-end low fidelity high fidelity design process html computer vision dsl 

摘      要：The way towards generating a website front end involves a designersettling on an idea for what kind of layout they want the website to have, thenproceeding to plan and implement each aspect one by one until they haveconverted what they initially laid out into its Html front end form, this processcan take a considerable time, especially considering the first draft of the designis traditionally never the final one. This process can take up a large amountof resource real estate, and as we have laid out in this paper, by using a Modelconsisting of various Neural Networks trained on a custom dataset. It can beautomated into assisting designers, allowing them to focus on the other morecomplicated parts of the system they are designing by quickly generating whatwould rather be straightforward busywork. Over the past 20 years, the boomin how much the internet is used and the sheer volume of pages on it demands ahigh level of work and time to create them. For the efficiency of the process, weproposed a multi-model-based architecture on image captioning, consisting ofConvolutional neural network (CNN) and Long short-term memory (LSTM)models. Our proposed approach trained on our custom-made database can beautomated into assisting designers, allowing them to focus on the other morecomplicated part of the system. We trained our model in several batches overa custom-made dataset consisting of over 6300 files and were finally able toachieve a Bilingual Evaluation Understudy (BLEU) score for a batch of 50hand-drawn images at 87.86%.