众包模式下基于参与者胜任度和接受度的任务推送模型
|
摘要
任务推送是众包模式下任务执行的重要方式,对提升任务完成质量以及提高参与者和发包方满意度具有积极影响。然而,当前研究将参与者胜任度作为确定合适参与者的主要指标,较少考虑参与者接受度对任务推送的影响。基于此,建立综合考虑参与者胜任度和接受度的任务推送模型,在模型中提出识别潜在参与者的思路和过程,利用启发式相似度算法测量任务相似度,描述并量化参与者表现,进而提出参与者胜任度测量方法。同时,基于参与者以往参与任务的特点,利用扩展的粗数方法,确定参与者接受度,并综合参与者胜任度确定潜在参与者的优先序。以一品威客平台为例,说明任务推送模型和方法的应用过程和有效性。研究结果表明,提出的模型和方法能够有效弥补常用的自选择任务执行方式的不足,避免参与者从大规模任务列表中选择合适任务,缩短任务的平均完成时间,同时能保障一定的参与者数量,为发包方提供多个较高质量的备选方案。与仅考虑参与者胜任度的任务推送相比,可以将任务推送给能够胜任且愿意接受任务的参与者,在一定程度上能够保证参与者的回复率,提高任务推送的成功率,减少多次推送以及由此带来的成本和时间的增加。研究结果丰富了众包模式下参与者胜任度和接受度的测量方法,扩展了众包任务推送领域的研究成果,为后续解决多任务指派优化问题奠定基础。实践中,参与者应当多参与任务且具有较好的表现,才能进一步提高后续获得合适推送任务的可能性。同时,对一些重要且要求较高的任务,发包方可采用任务推送方式完成该任务。
In crowdsourcing systems,task recommendation is an important way to perform tasks. It has positive impacts on the quality of task completion and participants' and requesters' satisfaction degree. However,current studies have mainly concentrated on whether participants complete the tasks needs to be recommended or not,but neglected whether they accept the recommended tasks.To overcome the weakness mentioned above,this paper constructs a model for task recommendation in crowdsourcing considering participants' competency and acceptance degree for a new task. In the model,we firstly proposed an idea to identify the alternative participants for the task needs to be recommended. By using the heuristic similarity measure approach,the similarity between the new task and its similar tasks are computed. Meanwhile,this paper describes and measures quantitatively participants' behaviors and performance on the tasks that they have contributed to. Then,the similarity between the recommended task and its similar tasks and participants' performance on these similar tasks are aggregated to determine their competency. Additionally,on the basis of tasks characteristics that a participant have involved in,this paper proposes an approach to determine his acceptance degree for the recommended task using the expanded rough number method. Finally,to demonstrate the implementation process and performance of the proposed approach,an illustrative example is conducted on epwk. com,a widely used Chinese online crowdsourcing market.The experimental results show that the proposed model and approach in this paper can fill the gap brought by the self-selection principle,which is a common way of performing tasks posted in crowdsourcing. It is also helpful to shorten the average time of completing the tasks,and to avoid selecting the proper tasks from a large number of alternative tasks for participants. Moreover,the model contributes to assure the number of participants who would participate in the recommended task and provide a few alternative solutions with high quality for requesters. In addition,comparing with task recommendation considering participants' competency solely,the proposed model and approach would be better in terms of participants' response rate. Furthermore,it promotes the success rate of task recommendation and decreases the probability of repeated recommendation that would lead to cost much more money and time.The study enriches approaches of determining participants' competency and acceptance degree for a task needs to be recommended,and expands research outcomes in the domain of task recommendation in crowdsourcing. Additionally,the research results in this paper establish the foundation for further solving the optimization problem of multi-task assignment. On the other hand,our study has implications to both participants and requesters. Hence,participants contribute to much more tasks and have high performance,then the possibility that they receive the proper recommeded tasks would be larger. For requesters,they would adopt the principle of task recommendation rather than self-selection principle to perform their posted tasks,especially for the tasks with great importance and requirement.
In crowdsourcing systems,task recommendation is an important way to perform tasks. It has positive impacts on the quality of task completion and participants' and requesters' satisfaction degree. However,current studies have mainly concentrated on whether participants complete the tasks needs to be recommended or not,but neglected whether they accept the recommended tasks.To overcome the weakness mentioned above,this paper constructs a model for task recommendation in crowdsourcing considering participants' competency and acceptance degree for a new task. In the model,we firstly proposed an idea to identify the alternative participants for the task needs to be recommended. By using the heuristic similarity measure approach,the similarity between the new task and its similar tasks are computed. Meanwhile,this paper describes and measures quantitatively participants' behaviors and performance on the tasks that they have contributed to. Then,the similarity between the recommended task and its similar tasks and participants' performance on these similar tasks are aggregated to determine their competency. Additionally,on the basis of tasks characteristics that a participant have involved in,this paper proposes an approach to determine his acceptance degree for the recommended task using the expanded rough number method. Finally,to demonstrate the implementation process and performance of the proposed approach,an illustrative example is conducted on epwk. com,a widely used Chinese online crowdsourcing market.The experimental results show that the proposed model and approach in this paper can fill the gap brought by the self-selection principle,which is a common way of performing tasks posted in crowdsourcing. It is also helpful to shorten the average time of completing the tasks,and to avoid selecting the proper tasks from a large number of alternative tasks for participants. Moreover,the model contributes to assure the number of participants who would participate in the recommended task and provide a few alternative solutions with high quality for requesters. In addition,comparing with task recommendation considering participants' competency solely,the proposed model and approach would be better in terms of participants' response rate. Furthermore,it promotes the success rate of task recommendation and decreases the probability of repeated recommendation that would lead to cost much more money and time.The study enriches approaches of determining participants' competency and acceptance degree for a task needs to be recommended,and expands research outcomes in the domain of task recommendation in crowdsourcing. Additionally,the research results in this paper establish the foundation for further solving the optimization problem of multi-task assignment. On the other hand,our study has implications to both participants and requesters. Hence,participants contribute to much more tasks and have high performance,then the possibility that they receive the proper recommeded tasks would be larger. For requesters,they would adopt the principle of task recommendation rather than self-selection principle to perform their posted tasks,especially for the tasks with great importance and requirement.
引文
[1]王姝,陈劲,梁靓.网络众包模式的协同自组织创新效应分析.科研管理,2014,35(4):26-33.WANG Shu,CHEN Jin,LIANG Liang. Analysis of collaborative and self-organizing innovation effect of the network crowdsourcing model. Science Research Management,2014,35(4):26-33.(in Chinese)
[2] ESTELLS-AROLAS E,GONZLEZ-LADRN-DE-GUEVARA F. Towards an integrated crowdsourcing definition. Journal of Information Science,2012,38(2):189-200.
[3] SCHMIDT G B,JETTINGHOFF W M. Using Amazon mechanical turk and other compensated crowdsourcing sites.Business Horizons,2016,59(4):391-400.
[4]董坤祥,侯文华,周常宝,等.众包竞赛中解答者创新绩效影响因素研究:感知风险的调节效应.科学学与科学技术管理,2016,37(2):21-29.DONG Kunxiang,HOU Wenhua,ZHOU Changbao,et al.Factors affecting innovation performance of solvers in crowdsourcing contest:the moderating role of perceived risk.Science of Science and Management of S.&T.,2016,37(2):21-29.(in Chinese)
[5] GEIGER D,SCHADER M. Personalized task recommendation in crowdsourcing information systems:current state of the art.Decision Support Systems,2014,65(C):3-16.
[6]郭伟,邱丹逸.基于多Agent的众包任务推荐系统建模与仿真.计算机工程与科学,2017,39(5):958-963.GUO Wei,QIU Danyi. Multi-agent based modeling and simulation of crowdsourcing task recommendation system. Computer Engineering&Science,2017,39(5):958-963.(in Chinese)
[7] ALSAYASNEH M,AMER-YAHIA S,GAUSSIER E,et al.Personalized and diverse task composition in crowdsourcing.IEEE Transactions on Knowledge and Data Engineering,2018,30(1):128-141.
[8]张光前,白雪.基于消费性格的新商品推荐方法.管理科学,2015,28(2):60-68.ZHANG Guangqian,BAI Xue. Method of new commodities recommendation based on consuming personalities. Journal of Management Science,2015,28(2):60-68.(in Chinese)
[9] GONG Y. Estimating participants for knowledge-intensive tasks in a network of crowdsourcing marketplaces. Information Systems Frontiers,2017,19(2):301-319.
[10] ROY S B,LYKOURENTZOU I,THIRUMURUGANATHAN S,et al. Task assignment optimization in knowledge-intensive crowdsourcing. The VLDB Journal,2015,24(4):467-491.
[11]郝琳娜,侯文华,张李浩,等.基于众包虚拟社区的诚信保障和信誉评价机制研究.系统工程理论与实践,2014,34(11):2837-2848.HAO Linna,HOU Wenhua,ZHANG Lihao,et al. Research on integrity safeguard mechanism and reputation evaluation mechanism of crowdsourcing in the virtual community. Systems Engineering-Theory&Practice,2014,34(11):2837-2848.(in Chinese)
[12] ZHENG H C,LI D H,HOU W H. Task design,motivation,and participation in crowdsourcing contests. International Journal of Electronic Commerce,2011,15(4):57-88.
[13] YUEN M C,KING I,LEUNG K S. Taskrec:a task recommendation framework in crowdsourcing systems. Neural Processing Letters,2015,41(2):223-238.
[14] SATZGER B,PSAIER H,SCHALL D,et al. Stimulating skill evolution in market-based crowdsourcing. Business Process Management,2011,6896(5):66-82.
[15]刘景方,张朋柱,吕英杰,等.基于文本挖掘的众包人才能力分析.系统管理学报,2015,24(3):365-371.LIU Jingfang,ZHANG Pengzhu,LYU Yingjie,et al. Analysis of the competence of crowdsourcing talents using text mining.Journal of Systems&Management,2015,24(3):365-371.(in Chinese)
[16]BERKOVSKY S,KUFLIK T,RICCI F. Mediation of user models for enhanced personalization in recommender systems. User Modeling and User-Adapted Interaction,2008,18(3):245-286.
[17]沈洪洲.基于胜任力的应急信息管理众包参与者选择模型研究.图书馆学研究,2017(13):93-101.SHEN Hongzhou. A research of selection model for crowdsourcing participants in competency-based emergency information management. Research on Library Science,2017(13):93-101.(in Chinese)
[18]吕英杰,张朋柱,刘景方.众包模式中面向创新任务的知识型人才选择.系统管理学报,2013,22(1):60-66.LYU Yingjie,ZHANG Pengzhu,LIU Jingfang. Task-oriented talent selection in crowdsouring. Journal of Systems&Management,2013,22(1):60-66.(in Chinese)
[19]朱小宁.支持任务推送的众包系统的研究与实现.北京:北京邮电大学,2015:53-65.ZHU Xiaoning. Research and implementation of a crowdsourcing system supporting task routing. Beijing:Beijing University of Posts and Telecommunications,2015:53-65.(in Chinese)
[20]黄丽媚.基于众包平台的用户信用度及消息推送算法研究.广州:广东技术师范学院,2016:9-30.HUANG Limei. Research on the user credit and message push algorithm on crowdsouring platform. Guangzhou:Guangdong Polytechnic Normal University,2016:9-30.(in Chinese)
[21] KARGER D R,OH S,SHAH D. Budget-optimal task allocation for reliable crowdsourcing systems. Operations Research,2013,62(1):1-24.
[22] BABA Y,KINOSHITA K,KASHIMA H. Participation recommendation system for crowdsourcing contests. Expert Systems with Applications,2016,58(C):174-183.
[23] SATZGER B,PSAIER H,SCHALL D,et al. Auction-based crowdsourcing supporting skill management. Information Systems,2013,38(4):547-560.
[24] ZHAO D,LI X Y,MA H. Budget-feasible online incentive mechanisms for crowdsourcing tasks truthfully. IEEE/ACM Transactions on Networking,2016,24(2):647-661.
[25] HERNANDO A,BOBADILLA J,ORTEGA F,et al. A probabilistic model for recommending to new cold-start non-registered users. Information Sciences,2017,376:216-232.
[26] ALEKSANDROVA M,BRUN A,BOYER A,et al. Identifying representative users in matrix factorization-based recommender systems:application to solving the content-less new item cold-start problem. Journal of Intelligent Information Systems,2017,48(2):365-397.
[27] PELNEK R. Bayesian knowledge tracing,logistic models,and beyond:an overview of learner modeling techniques. User Modeling and User-Adapted Interaction,2017,27(3/5):313-350.
[28]蒋国银,胡斌.协同工作下角色-任务匹配与互动多智能体模拟研究.管理科学,2009,22(6):56-63.JIANG Guoyin,HU Bin. Agent-based simulation for matching and interaction between roles and tasks under collaborative work environment. Journal of Management Science,2009,22(6):56-63.(in Chinese)
[29] JIANG H,QI X,SUN H. Choice-based recommender systems:a unified approach to achieving relevancy and diversity. Operations Research,2014,62(5):973-993.
[30] SCHENK E,GUITTARD C. Towards a characterization of crowdsourcing practices. Journal of Innovation Economics&Management,2011,1(7):93-107.
[31] JIANG H. A nested logit-based approach to measuring air shopping screen quality and predicting market share. Journal of Revenue&Pricing Management,2009,8(2/3):134-147.
[32] XU Y,BERNARD A,PERRY N,et al. Knowledge evaluation in product lifecycle design and support. KnowledgeBased Systems,2014,70:256-267.
[33]王文宪,陈钉钧,吕红霞,等.基于Nested Logit模型的铁路旅客客运产品选择研究.交通运输系统工程与信息,2015,15(4):140-146.WANG Wenxian,CHEN Dingjun,LYU Hongxia,et al. Research of transport product choice for railway passengers based on nested logit model. Journal of Transportation Systems Engineering and Information Technology,2015,15(4):140-146.(in Chinese)
[34] AHN H J. A new similarity measure for collaborative filtering to alleviate the new user cold-starting problem. Information Sciences,2008,178(1):37-51.
[35]王刚,钟国祥.一种基于本体相似度计算的文本聚类算法研究.计算机科学,2010,37(9):222-224,228.WANG Gang,ZHONG Guoxiang. Study on text clustering algorithm based on similarity measurement of ontology. Computer Science,2010,37(9):222-224,228.(in Chinese)
[36] KIM H N,HA I,LEE K S,et al. Collaborative user modeling for enhanced content filtering in recommender systems.Decision Support Systems,2011,51(4):772-781.
[37] HADI-VENCHEH A,MOHAMADGHASEMI A. A fuzzy AHPDEA approach for multiple criteria ABC inventory classification. Expert Systems with Applications,2011,38(4):3346-3352.
[38] ZHANG X,LIU J J,COLE M,et al. Predicting users'domain knowledge in information retrieval using multiple regression analysis of search behaviors. Journal of the Association for Information Science and Technology,2014,66(5):980-1000.
[39]SCHOLZ M,DORNER V,FRANZ M,et al. Measuring consumers'willingness to pay with utility-based recommendation systems. Decision Support Systems,2015,72(C):60-71.
[40]赵文燕,张换高,何桢,等.粗数:一种客户需求分析方法.计算机集成制造系统,2011,17(11):2493-2501.ZHAO Wenyan,ZHANG Huangao,HE Zhen,et al. Rough number:customer requirment analytical method. Computer Integrated Manufacturing Systems,2011,17(11):2493-2501.(in Chinese)
[41] ZHAI L Y,KHOO L P,ZHONG Z W. A rough set enhanced fuzzy approach to quality function deployment. The International Journal of Advanced Manufacturing Technology,2008,37(5/6):613-624.
[42] TERWIESCH C,XU Y. Innovation contests,open innovation,and multiagent problem solving. Management Science,2008,54(9):1529-1543.
[2] ESTELLS-AROLAS E,GONZLEZ-LADRN-DE-GUEVARA F. Towards an integrated crowdsourcing definition. Journal of Information Science,2012,38(2):189-200.
[3] SCHMIDT G B,JETTINGHOFF W M. Using Amazon mechanical turk and other compensated crowdsourcing sites.Business Horizons,2016,59(4):391-400.
[4]董坤祥,侯文华,周常宝,等.众包竞赛中解答者创新绩效影响因素研究:感知风险的调节效应.科学学与科学技术管理,2016,37(2):21-29.DONG Kunxiang,HOU Wenhua,ZHOU Changbao,et al.Factors affecting innovation performance of solvers in crowdsourcing contest:the moderating role of perceived risk.Science of Science and Management of S.&T.,2016,37(2):21-29.(in Chinese)
[5] GEIGER D,SCHADER M. Personalized task recommendation in crowdsourcing information systems:current state of the art.Decision Support Systems,2014,65(C):3-16.
[6]郭伟,邱丹逸.基于多Agent的众包任务推荐系统建模与仿真.计算机工程与科学,2017,39(5):958-963.GUO Wei,QIU Danyi. Multi-agent based modeling and simulation of crowdsourcing task recommendation system. Computer Engineering&Science,2017,39(5):958-963.(in Chinese)
[7] ALSAYASNEH M,AMER-YAHIA S,GAUSSIER E,et al.Personalized and diverse task composition in crowdsourcing.IEEE Transactions on Knowledge and Data Engineering,2018,30(1):128-141.
[8]张光前,白雪.基于消费性格的新商品推荐方法.管理科学,2015,28(2):60-68.ZHANG Guangqian,BAI Xue. Method of new commodities recommendation based on consuming personalities. Journal of Management Science,2015,28(2):60-68.(in Chinese)
[9] GONG Y. Estimating participants for knowledge-intensive tasks in a network of crowdsourcing marketplaces. Information Systems Frontiers,2017,19(2):301-319.
[10] ROY S B,LYKOURENTZOU I,THIRUMURUGANATHAN S,et al. Task assignment optimization in knowledge-intensive crowdsourcing. The VLDB Journal,2015,24(4):467-491.
[11]郝琳娜,侯文华,张李浩,等.基于众包虚拟社区的诚信保障和信誉评价机制研究.系统工程理论与实践,2014,34(11):2837-2848.HAO Linna,HOU Wenhua,ZHANG Lihao,et al. Research on integrity safeguard mechanism and reputation evaluation mechanism of crowdsourcing in the virtual community. Systems Engineering-Theory&Practice,2014,34(11):2837-2848.(in Chinese)
[12] ZHENG H C,LI D H,HOU W H. Task design,motivation,and participation in crowdsourcing contests. International Journal of Electronic Commerce,2011,15(4):57-88.
[13] YUEN M C,KING I,LEUNG K S. Taskrec:a task recommendation framework in crowdsourcing systems. Neural Processing Letters,2015,41(2):223-238.
[14] SATZGER B,PSAIER H,SCHALL D,et al. Stimulating skill evolution in market-based crowdsourcing. Business Process Management,2011,6896(5):66-82.
[15]刘景方,张朋柱,吕英杰,等.基于文本挖掘的众包人才能力分析.系统管理学报,2015,24(3):365-371.LIU Jingfang,ZHANG Pengzhu,LYU Yingjie,et al. Analysis of the competence of crowdsourcing talents using text mining.Journal of Systems&Management,2015,24(3):365-371.(in Chinese)
[16]BERKOVSKY S,KUFLIK T,RICCI F. Mediation of user models for enhanced personalization in recommender systems. User Modeling and User-Adapted Interaction,2008,18(3):245-286.
[17]沈洪洲.基于胜任力的应急信息管理众包参与者选择模型研究.图书馆学研究,2017(13):93-101.SHEN Hongzhou. A research of selection model for crowdsourcing participants in competency-based emergency information management. Research on Library Science,2017(13):93-101.(in Chinese)
[18]吕英杰,张朋柱,刘景方.众包模式中面向创新任务的知识型人才选择.系统管理学报,2013,22(1):60-66.LYU Yingjie,ZHANG Pengzhu,LIU Jingfang. Task-oriented talent selection in crowdsouring. Journal of Systems&Management,2013,22(1):60-66.(in Chinese)
[19]朱小宁.支持任务推送的众包系统的研究与实现.北京:北京邮电大学,2015:53-65.ZHU Xiaoning. Research and implementation of a crowdsourcing system supporting task routing. Beijing:Beijing University of Posts and Telecommunications,2015:53-65.(in Chinese)
[20]黄丽媚.基于众包平台的用户信用度及消息推送算法研究.广州:广东技术师范学院,2016:9-30.HUANG Limei. Research on the user credit and message push algorithm on crowdsouring platform. Guangzhou:Guangdong Polytechnic Normal University,2016:9-30.(in Chinese)
[21] KARGER D R,OH S,SHAH D. Budget-optimal task allocation for reliable crowdsourcing systems. Operations Research,2013,62(1):1-24.
[22] BABA Y,KINOSHITA K,KASHIMA H. Participation recommendation system for crowdsourcing contests. Expert Systems with Applications,2016,58(C):174-183.
[23] SATZGER B,PSAIER H,SCHALL D,et al. Auction-based crowdsourcing supporting skill management. Information Systems,2013,38(4):547-560.
[24] ZHAO D,LI X Y,MA H. Budget-feasible online incentive mechanisms for crowdsourcing tasks truthfully. IEEE/ACM Transactions on Networking,2016,24(2):647-661.
[25] HERNANDO A,BOBADILLA J,ORTEGA F,et al. A probabilistic model for recommending to new cold-start non-registered users. Information Sciences,2017,376:216-232.
[26] ALEKSANDROVA M,BRUN A,BOYER A,et al. Identifying representative users in matrix factorization-based recommender systems:application to solving the content-less new item cold-start problem. Journal of Intelligent Information Systems,2017,48(2):365-397.
[27] PELNEK R. Bayesian knowledge tracing,logistic models,and beyond:an overview of learner modeling techniques. User Modeling and User-Adapted Interaction,2017,27(3/5):313-350.
[28]蒋国银,胡斌.协同工作下角色-任务匹配与互动多智能体模拟研究.管理科学,2009,22(6):56-63.JIANG Guoyin,HU Bin. Agent-based simulation for matching and interaction between roles and tasks under collaborative work environment. Journal of Management Science,2009,22(6):56-63.(in Chinese)
[29] JIANG H,QI X,SUN H. Choice-based recommender systems:a unified approach to achieving relevancy and diversity. Operations Research,2014,62(5):973-993.
[30] SCHENK E,GUITTARD C. Towards a characterization of crowdsourcing practices. Journal of Innovation Economics&Management,2011,1(7):93-107.
[31] JIANG H. A nested logit-based approach to measuring air shopping screen quality and predicting market share. Journal of Revenue&Pricing Management,2009,8(2/3):134-147.
[32] XU Y,BERNARD A,PERRY N,et al. Knowledge evaluation in product lifecycle design and support. KnowledgeBased Systems,2014,70:256-267.
[33]王文宪,陈钉钧,吕红霞,等.基于Nested Logit模型的铁路旅客客运产品选择研究.交通运输系统工程与信息,2015,15(4):140-146.WANG Wenxian,CHEN Dingjun,LYU Hongxia,et al. Research of transport product choice for railway passengers based on nested logit model. Journal of Transportation Systems Engineering and Information Technology,2015,15(4):140-146.(in Chinese)
[34] AHN H J. A new similarity measure for collaborative filtering to alleviate the new user cold-starting problem. Information Sciences,2008,178(1):37-51.
[35]王刚,钟国祥.一种基于本体相似度计算的文本聚类算法研究.计算机科学,2010,37(9):222-224,228.WANG Gang,ZHONG Guoxiang. Study on text clustering algorithm based on similarity measurement of ontology. Computer Science,2010,37(9):222-224,228.(in Chinese)
[36] KIM H N,HA I,LEE K S,et al. Collaborative user modeling for enhanced content filtering in recommender systems.Decision Support Systems,2011,51(4):772-781.
[37] HADI-VENCHEH A,MOHAMADGHASEMI A. A fuzzy AHPDEA approach for multiple criteria ABC inventory classification. Expert Systems with Applications,2011,38(4):3346-3352.
[38] ZHANG X,LIU J J,COLE M,et al. Predicting users'domain knowledge in information retrieval using multiple regression analysis of search behaviors. Journal of the Association for Information Science and Technology,2014,66(5):980-1000.
[39]SCHOLZ M,DORNER V,FRANZ M,et al. Measuring consumers'willingness to pay with utility-based recommendation systems. Decision Support Systems,2015,72(C):60-71.
[40]赵文燕,张换高,何桢,等.粗数:一种客户需求分析方法.计算机集成制造系统,2011,17(11):2493-2501.ZHAO Wenyan,ZHANG Huangao,HE Zhen,et al. Rough number:customer requirment analytical method. Computer Integrated Manufacturing Systems,2011,17(11):2493-2501.(in Chinese)
[41] ZHAI L Y,KHOO L P,ZHONG Z W. A rough set enhanced fuzzy approach to quality function deployment. The International Journal of Advanced Manufacturing Technology,2008,37(5/6):613-624.
[42] TERWIESCH C,XU Y. Innovation contests,open innovation,and multiagent problem solving. Management Science,2008,54(9):1529-1543.