基于GUI软件的测试自动化框架的研究与应用
摘要
图形用户界面(GUI)越来越广泛的在软件系统中使用。基于GUI软件的测试也变得日益重要。手工测试让测试人员感到单调乏味,成本高。随着GUI软件规模的扩大和软件复杂度的提高,测试工作越来越繁重,仅使用GUI的手工测试已经不能满足要求。因此,基于GUI软件的测试自动化势在必行。
目前业界普遍使用的是商用GUI软件自动化测试工具来实现测试自动化。但这些工具均采用捕获/回放机制,该机制没有对GUI软件测试的自动化提供很好的支持。而如何在现有测试工具上大幅度提升测试效率方面的研究也相对较少。因此对GUI软件的测试自动化的研究具有实用价值。
本文较为系统地分析了GUI软件的测试自动化技术的研究现状,结合测试自动化理论和技术,提出了一种比较完整有效的基于GUI软件的测试自动化框架(GSTAF4SEI)。该测试框架不仅支持GUI的C/S(client/server)软件测试,另外还支持当前Web应用程序测试。其中重点探讨了该框架需要解决的三个关键技术:测试用例生成、测试结果验证和测试脚本设计与执行。使用现有的基于GUI软件的测试自动化工具,将该框架应用于Web应用程序“煤质管理信息平台”的测试中,对本文提出的测试自动化解决方案具有的实用价值进行了验证。
The widespread recognition of the usefulness of graphical user interface (GUI) has established their importance as critical components of today's software.GUI manual testing makes testers dull and has high-cost. With the wide software size and improve of software complexity, the workload of software testing has more and more heavy. It's not satisfied with GUI software manual testing lonely. So it is imperative to automated testing of GUI software.
There exists many commercial tools for testing GUI software, which are Capture-Playback mechanism-based.However, they haven't provided enough support for automated testing of GUI software.Therefore it is necessary and meaningful to research on automated testing.
This dissertation systematically analyzes the research status of graphical user interfaces. According to the presented theoy and technique of testing automation, It proposes a complete and efficient GUI Software Testing Automation Framework (GSTAF4SEI) .This framework not only supports C/S (client/server) software based on GUI testing , but also supports web application.
This dissertation researches three key technologies based on that framework. They are test cases generation, the result of automated comparison, test script design and execution. Using automated testing tool based on GUI software called RFT, The framework applies in the testing of web application called Coal Quality Management Information System (CQMIS).The application value of the solution to automated testing is verified consequently.
目前业界普遍使用的是商用GUI软件自动化测试工具来实现测试自动化。但这些工具均采用捕获/回放机制,该机制没有对GUI软件测试的自动化提供很好的支持。而如何在现有测试工具上大幅度提升测试效率方面的研究也相对较少。因此对GUI软件的测试自动化的研究具有实用价值。
本文较为系统地分析了GUI软件的测试自动化技术的研究现状,结合测试自动化理论和技术,提出了一种比较完整有效的基于GUI软件的测试自动化框架(GSTAF4SEI)。该测试框架不仅支持GUI的C/S(client/server)软件测试,另外还支持当前Web应用程序测试。其中重点探讨了该框架需要解决的三个关键技术:测试用例生成、测试结果验证和测试脚本设计与执行。使用现有的基于GUI软件的测试自动化工具,将该框架应用于Web应用程序“煤质管理信息平台”的测试中,对本文提出的测试自动化解决方案具有的实用价值进行了验证。
The widespread recognition of the usefulness of graphical user interface (GUI) has established their importance as critical components of today's software.GUI manual testing makes testers dull and has high-cost. With the wide software size and improve of software complexity, the workload of software testing has more and more heavy. It's not satisfied with GUI software manual testing lonely. So it is imperative to automated testing of GUI software.
There exists many commercial tools for testing GUI software, which are Capture-Playback mechanism-based.However, they haven't provided enough support for automated testing of GUI software.Therefore it is necessary and meaningful to research on automated testing.
This dissertation systematically analyzes the research status of graphical user interfaces. According to the presented theoy and technique of testing automation, It proposes a complete and efficient GUI Software Testing Automation Framework (GSTAF4SEI) .This framework not only supports C/S (client/server) software based on GUI testing , but also supports web application.
This dissertation researches three key technologies based on that framework. They are test cases generation, the result of automated comparison, test script design and execution. Using automated testing tool based on GUI software called RFT, The framework applies in the testing of web application called Coal Quality Management Information System (CQMIS).The application value of the solution to automated testing is verified consequently.
引文
[1]B.A.Myers.Userinterface software tools.ACM Transaction on Computer-Human Interaction[J].1995,2(1):64-103.
[2]Laura Rose.Getting the Most from Your Automated Testing[M].2004.
[3]Jeff Offutt,Shaoying Liu,Aynur Abdurazik.Generating Test Data From State-based Specifications.The Journal of Software Testing,Verification And Reliability,2003,13(1):25-53.
[4]Jeff Offutt,Yiwei Xiong,Shaoying Liu.Criteria for Generating Specification based Tests.Fifth IEEE International Conference on Engineering of Complex Computer Systems(ICECCS '99).Washington,DC:IEEE.CS Press,1999:119-131.
[5]Jeff Offutt,Aynur Abdurazik.Generating Tests from UML Specifications.Second International Conference on the Unified Modeling Language(UML99).1999:416-429.
[6]Jeff Offutt,Shaoying Liu.Generating Test Data from SOFL Specifications.The Journal of Systems and Software.1999,49(1):49-62.
[7]Jeff Offutt,Aynur Abdurazik.Using UML Collaboration Diagrams for Static Checking and Test Generation.The Third International Conference On the Unified Modeling Language(UML'00).2000:383-395.
[8]Aynur Abdurazik,Paul Ammann,Wei Ding.Evaluation of Three Specification-based Testing Criteria.Sixth IEEE International Conference on Engineering of Complex Computer Systems(ICECCS '00)Washington,DC:IEEE CS Press.2003:179-187.
[9]Atif.M.Memon.M.E.Pollack.M.L.Soffa.Hierarchical GUI Test CaseGeneration Using Automated Planning.IEEE Transactions on Software Engineering.2005,27(2):144-155.
[10]Atif.M.Memon,M.E.Pollack,M.L.Soffa.Automated Test Oracles for GUIs.Proceedings of the ACM SIGSOFT 8th International Symposium on the Foundations of Software Engineering(FSE-8).New York:2002,30-39.
[11]刘超.程序交互执行流程图及其测试覆盖准则.软件学报.1998,9(6):458-463.
[12]顾玉良,王立福.界面类对象建模技术研究.计算机工程。1999,25(7):21-23.
[13]Atif.M.Memon,Mary Lou Soffa.Regression testing of GUIs.Proceedings of the 9th European Software Engineering Conference held jointly with 10th ACM SIGSOFT.New York:ACM Press.2006:118-127.
[14]杜栓柱.基于界面构建关联图的软件功能测试技术.计算机研究与发展.2002, 39(2):148-152.
[15]姜卫,汪厚祥.图形用户界面的测试自动化[J].舰船电子工程,2004,24(3):50-52.
[16]姚实颖.软件测试自动化中建立可维护脚本的技术[J].计算机工程,2003,29(11):79-81.
[17]CQMIS项目组 淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台需求规格说明书V2.0文档编号:HGD-HBCQMIS-REQ.合肥工业大学软件工程研究室,2006.6.
[18]朱菊.软件自动化测试框架TAF及其应用[D].硕士毕业论文.河海大学,2006.
[19]Keith Zambelich.Totally Data-Driven Automated Testing.1998.
[20]Walworth A.Java GUI testing[J].Dr.Dobb's Journal,1997,(2):30-34.
[21]Fewster M.软件测试自动化技术与实例详解[M].舒智勇译.北京:电子工业出版社.2000,57-58.
[22]LK Ha,BW Kang.Meta-validation of UML structural diagrams and behavioral diagrams with consistency rules.IEEE Pacific Rim Conference on Computers and signal Processing,Victoria,2003:679-683.
[23]Nagle,Carl,Test Automation Frameworks,http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFrameworks.htm,2000.
[24]Peer,你的组织为自动化测试做好准备了吗?http://www-128.ibm.com/developerworks/cn/rational/r-testauto/,2005.
[25]Beizer B,Software Testing Techniques,2nd edition,Van Nostr and Reinhold,1990.
[26]Kaner C,Falk J,Nguyen H Q.Testing Computer Software[Second Edition].Wiley Computer Publishing,2000.
[27]Auslander D M,Tham C H.Real-Time Software for Control:Program Examples in C[M].Prentice Hall.Int,1990.
[28]Shim K G,Ramanatham P.Real-Time Computer Science and Engineering[C].Proceedings of the IEEE,1994,82(1):6-25.
[29]郑人杰.计算机软件测试技术[M].清华大学出版社,1992:16-25.
[30]Elfriede Dustin,Jeff Rashka,John Paul.Automated Software Testing:Introduction,Management,and Performance.Addison-Wesley Longman Publishing Co.,Inc.,Boston,MA,1999.
[31]Mark Utting,Alexauder Pretschner and Bruno Legard.A taxonomy of model-based testing working[EB/OL].http://www.cs.waikato.ac.nz/pubs/wp/2006/uow-cs-wp-2006-04.pdf,2006.4.
[32]Atif M.Memon.GUI testing:Pitfalls and process.IEEE Computer 35(8):90-91 2003.
[33] Atif M. Memon. A comprehensive framework for testing graphical user interfaces UMI Microform.2002.
[34] Kanglin Li, Mengqi Wu. Effective GUI Test Automation Developing an Automated GUI Testing Tool.Sybex. 2004.
[35] S. Rosaria, H. Robinson.Applying models in your testing process. Information and Software Technology 42(2000) 815-824.
[36] Sommervillel, SawyerP.Requirements Engineering: A Good Practice, Wiley, Chichester UK, 1997.
[37] IBM WebSphere Help System [EB/OL]. http://publib.boulder.ibm.com/infocenter/rtnlhelp/v6r0m0/index.jsp?topic=/com.r ational.test.ft.help/icwelcome_product_rft.htm&S_TACT=105AGX52&S_CMP =cn-a-r,2006.10.
[38] Chow T S. Testing design modeled by finite-state machines [J]. IEEE Transactions on Software Engineering, 1978.10, 4(3): 178-187.
[39] Fujiwara S, Bochmann G, Khendek F.Test selection based on finite state models [J]. IEEE Transactions on Software Engineering, 1991.9, 17(6):591-603.
[40] Beizer B.Black-Box Testing: Techniques for Functional Testing of Software and Systems, Wiley, New York, USA, 1995.
[41] Abdurazik A, Offutt J.Using UML collaboration diagrams for static checking and test generation.UML'OO, UK, 2000.
[42] Harel D.Statecharts: a visual formalism for complex systems, Science of Computer Programming, 1987, 8(3):231-274.
[43] Offutt J, Abdurazik A.Generating test cases from UML specifications.UML'99, USA, 1999.
[44] Hong H S,Kim Y G,Cha D.A test sequence selection method for statecharts,The Journal of Software Testing ,Verification&Reliability,2000,10(4):203-227.
[45] Vritzer A,Larson B.Load testing software using deterministic state testing .ISSTA'93,USA,1993.
[46] Whittaker J A, Thomason M G..A Markov chain model for statistical software testing.IEEE Transactions on Software Engineering, 1994, 20(10):812-824.
[47] Poore J H. Introduction to the special issue on: model-based statistical testing of software intensive systems, Information and Software Technology, 2000, 42(12):797-799.
[48] Maurer P M.The design and implementation of a grammar-based data generator.Software Practice&Experience, 1992,23(3):223-244.
[49]Lee D,Yannakakis M.Principle and methods of testing finite state machines-A survey[J].Proc.of the IEEE,1999,84(8):1090-1123.
[50]Robinson H.intelligent test automation[EB/OL].http://www.geocities.com/harry_robinson_testing/robinson.pdf,2006.10.
[51]Bret Pettichord.Hey vendors,give us real scripting languages[EB/OL].http://www.stickyminds.com/sitewide.asp?Objectld=2326&ObjectType=COL&F unction=edetail,2005.2.
[52]CQMIS项目组.淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台研制技术报告V2.0.文档编号:HGD-CQMIS-RTR.合肥工业大学软件工程研究室,2006.6.
[53]CQMIS项目组.淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台研制工作报告V2.0.文档编号:HGD-CQMIS-RWR.合肥工业大学软件工程研究室,2006.6.
[54]MichaelKelly.Choosing a test automation framework.http://www.ibm.com/developerworks/rational/library/591.html?STACT=105AGX52&S CMP=cn-a-r 2003.
[55]Bush,M."Improving Software Quality:The Use of Formal In sections at the Jet Propulsion L aboratory".Proceedings of the 12~(th)International Conference on Soft ware Engineering(ICSE12).Nice,France.1990,196-199.
[56]CQMIS项目组.淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台概要设计说明书V2.0.文档编号:HGD-HBCQMIS-DSN.合肥工业大学软件工程研究室,2006.6.
[57]徐宏喆.UML自动化测试技术[M].北京:西安交通大学出版社.2006,57-58.
[58]冯玉才,唐艳,周淳.关键字驱动自动化测试的原理和实现[J].计算机应用,2004,24(8):140-142.
[59]黄陇,陈致名.基于UML的软件测试自动化研究[J].计算机应用,2004,24(7):135-137.
[60]于秀山,于洪敏.软件测试新技术与实践[M].北京:电子工业出版社.2006,82-101.
[2]Laura Rose.Getting the Most from Your Automated Testing[M].2004.
[3]Jeff Offutt,Shaoying Liu,Aynur Abdurazik.Generating Test Data From State-based Specifications.The Journal of Software Testing,Verification And Reliability,2003,13(1):25-53.
[4]Jeff Offutt,Yiwei Xiong,Shaoying Liu.Criteria for Generating Specification based Tests.Fifth IEEE International Conference on Engineering of Complex Computer Systems(ICECCS '99).Washington,DC:IEEE.CS Press,1999:119-131.
[5]Jeff Offutt,Aynur Abdurazik.Generating Tests from UML Specifications.Second International Conference on the Unified Modeling Language(UML99).1999:416-429.
[6]Jeff Offutt,Shaoying Liu.Generating Test Data from SOFL Specifications.The Journal of Systems and Software.1999,49(1):49-62.
[7]Jeff Offutt,Aynur Abdurazik.Using UML Collaboration Diagrams for Static Checking and Test Generation.The Third International Conference On the Unified Modeling Language(UML'00).2000:383-395.
[8]Aynur Abdurazik,Paul Ammann,Wei Ding.Evaluation of Three Specification-based Testing Criteria.Sixth IEEE International Conference on Engineering of Complex Computer Systems(ICECCS '00)Washington,DC:IEEE CS Press.2003:179-187.
[9]Atif.M.Memon.M.E.Pollack.M.L.Soffa.Hierarchical GUI Test CaseGeneration Using Automated Planning.IEEE Transactions on Software Engineering.2005,27(2):144-155.
[10]Atif.M.Memon,M.E.Pollack,M.L.Soffa.Automated Test Oracles for GUIs.Proceedings of the ACM SIGSOFT 8th International Symposium on the Foundations of Software Engineering(FSE-8).New York:2002,30-39.
[11]刘超.程序交互执行流程图及其测试覆盖准则.软件学报.1998,9(6):458-463.
[12]顾玉良,王立福.界面类对象建模技术研究.计算机工程。1999,25(7):21-23.
[13]Atif.M.Memon,Mary Lou Soffa.Regression testing of GUIs.Proceedings of the 9th European Software Engineering Conference held jointly with 10th ACM SIGSOFT.New York:ACM Press.2006:118-127.
[14]杜栓柱.基于界面构建关联图的软件功能测试技术.计算机研究与发展.2002, 39(2):148-152.
[15]姜卫,汪厚祥.图形用户界面的测试自动化[J].舰船电子工程,2004,24(3):50-52.
[16]姚实颖.软件测试自动化中建立可维护脚本的技术[J].计算机工程,2003,29(11):79-81.
[17]CQMIS项目组 淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台需求规格说明书V2.0文档编号:HGD-HBCQMIS-REQ.合肥工业大学软件工程研究室,2006.6.
[18]朱菊.软件自动化测试框架TAF及其应用[D].硕士毕业论文.河海大学,2006.
[19]Keith Zambelich.Totally Data-Driven Automated Testing.1998.
[20]Walworth A.Java GUI testing[J].Dr.Dobb's Journal,1997,(2):30-34.
[21]Fewster M.软件测试自动化技术与实例详解[M].舒智勇译.北京:电子工业出版社.2000,57-58.
[22]LK Ha,BW Kang.Meta-validation of UML structural diagrams and behavioral diagrams with consistency rules.IEEE Pacific Rim Conference on Computers and signal Processing,Victoria,2003:679-683.
[23]Nagle,Carl,Test Automation Frameworks,http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFrameworks.htm,2000.
[24]Peer,你的组织为自动化测试做好准备了吗?http://www-128.ibm.com/developerworks/cn/rational/r-testauto/,2005.
[25]Beizer B,Software Testing Techniques,2nd edition,Van Nostr and Reinhold,1990.
[26]Kaner C,Falk J,Nguyen H Q.Testing Computer Software[Second Edition].Wiley Computer Publishing,2000.
[27]Auslander D M,Tham C H.Real-Time Software for Control:Program Examples in C[M].Prentice Hall.Int,1990.
[28]Shim K G,Ramanatham P.Real-Time Computer Science and Engineering[C].Proceedings of the IEEE,1994,82(1):6-25.
[29]郑人杰.计算机软件测试技术[M].清华大学出版社,1992:16-25.
[30]Elfriede Dustin,Jeff Rashka,John Paul.Automated Software Testing:Introduction,Management,and Performance.Addison-Wesley Longman Publishing Co.,Inc.,Boston,MA,1999.
[31]Mark Utting,Alexauder Pretschner and Bruno Legard.A taxonomy of model-based testing working[EB/OL].http://www.cs.waikato.ac.nz/pubs/wp/2006/uow-cs-wp-2006-04.pdf,2006.4.
[32]Atif M.Memon.GUI testing:Pitfalls and process.IEEE Computer 35(8):90-91 2003.
[33] Atif M. Memon. A comprehensive framework for testing graphical user interfaces UMI Microform.2002.
[34] Kanglin Li, Mengqi Wu. Effective GUI Test Automation Developing an Automated GUI Testing Tool.Sybex. 2004.
[35] S. Rosaria, H. Robinson.Applying models in your testing process. Information and Software Technology 42(2000) 815-824.
[36] Sommervillel, SawyerP.Requirements Engineering: A Good Practice, Wiley, Chichester UK, 1997.
[37] IBM WebSphere Help System [EB/OL]. http://publib.boulder.ibm.com/infocenter/rtnlhelp/v6r0m0/index.jsp?topic=/com.r ational.test.ft.help/icwelcome_product_rft.htm&S_TACT=105AGX52&S_CMP =cn-a-r,2006.10.
[38] Chow T S. Testing design modeled by finite-state machines [J]. IEEE Transactions on Software Engineering, 1978.10, 4(3): 178-187.
[39] Fujiwara S, Bochmann G, Khendek F.Test selection based on finite state models [J]. IEEE Transactions on Software Engineering, 1991.9, 17(6):591-603.
[40] Beizer B.Black-Box Testing: Techniques for Functional Testing of Software and Systems, Wiley, New York, USA, 1995.
[41] Abdurazik A, Offutt J.Using UML collaboration diagrams for static checking and test generation.UML'OO, UK, 2000.
[42] Harel D.Statecharts: a visual formalism for complex systems, Science of Computer Programming, 1987, 8(3):231-274.
[43] Offutt J, Abdurazik A.Generating test cases from UML specifications.UML'99, USA, 1999.
[44] Hong H S,Kim Y G,Cha D.A test sequence selection method for statecharts,The Journal of Software Testing ,Verification&Reliability,2000,10(4):203-227.
[45] Vritzer A,Larson B.Load testing software using deterministic state testing .ISSTA'93,USA,1993.
[46] Whittaker J A, Thomason M G..A Markov chain model for statistical software testing.IEEE Transactions on Software Engineering, 1994, 20(10):812-824.
[47] Poore J H. Introduction to the special issue on: model-based statistical testing of software intensive systems, Information and Software Technology, 2000, 42(12):797-799.
[48] Maurer P M.The design and implementation of a grammar-based data generator.Software Practice&Experience, 1992,23(3):223-244.
[49]Lee D,Yannakakis M.Principle and methods of testing finite state machines-A survey[J].Proc.of the IEEE,1999,84(8):1090-1123.
[50]Robinson H.intelligent test automation[EB/OL].http://www.geocities.com/harry_robinson_testing/robinson.pdf,2006.10.
[51]Bret Pettichord.Hey vendors,give us real scripting languages[EB/OL].http://www.stickyminds.com/sitewide.asp?Objectld=2326&ObjectType=COL&F unction=edetail,2005.2.
[52]CQMIS项目组.淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台研制技术报告V2.0.文档编号:HGD-CQMIS-RTR.合肥工业大学软件工程研究室,2006.6.
[53]CQMIS项目组.淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台研制工作报告V2.0.文档编号:HGD-CQMIS-RWR.合肥工业大学软件工程研究室,2006.6.
[54]MichaelKelly.Choosing a test automation framework.http://www.ibm.com/developerworks/rational/library/591.html?STACT=105AGX52&S CMP=cn-a-r 2003.
[55]Bush,M."Improving Software Quality:The Use of Formal In sections at the Jet Propulsion L aboratory".Proceedings of the 12~(th)International Conference on Soft ware Engineering(ICSE12).Nice,France.1990,196-199.
[56]CQMIS项目组.淮北矿业(集团)有限公司袁庄煤矿煤质管理信息平台概要设计说明书V2.0.文档编号:HGD-HBCQMIS-DSN.合肥工业大学软件工程研究室,2006.6.
[57]徐宏喆.UML自动化测试技术[M].北京:西安交通大学出版社.2006,57-58.
[58]冯玉才,唐艳,周淳.关键字驱动自动化测试的原理和实现[J].计算机应用,2004,24(8):140-142.
[59]黄陇,陈致名.基于UML的软件测试自动化研究[J].计算机应用,2004,24(7):135-137.
[60]于秀山,于洪敏.软件测试新技术与实践[M].北京:电子工业出版社.2006,82-101.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |