无线传感器网络分布式调度方法研究
|
摘要
无线传感器网络本质上是一类资源受限的网络。通常情况下,无线传感器网络节点采用电池供电,有限的能量限制了网络生存期。作为一个嵌入式系统,节点的计算能力和存储能力都较小;节点间通信带宽也比较低。因此,在这些约束条件下,无线传感器网络调度方法对于WSN应用中网络性能的提升具有极其重要的意义。
本文对无线传感器网络分布式调度方法进行了深入的研究,并针对无线传感器网络能量受限的特点,提出了四个分布式调度方法。论文的主要贡献和创新点包括:
(1)提出了一类实用的、协作分布式的调度方法。该类方法包含两种调度方法:一步协作分布式调度方法和两步协作分布式调度方法。该类方法基于马尔可夫链,使节点通过相互协作学习其它节点的行为信息,将节点传输调度与休眠调度相结合,达到节省能量的目的。分析了该类方法的适应性、实用性,并从理论上证明了其收敛性。实验结果表明这两个调度方法能够有效地减少能量消耗。
(2)提出了一种WSN分布式自学习调度方法。该方法在WSN分布式调度研究中引入再励学习的思想,通过对Q-学习方法扩展,提出了一种计算调度参数的近似方法,使节点可以得到具有连续值空间的发送调度参数和休眠调度参数,从而使节点实现传输调度和休眠调度。该方法在优化节点能耗的同时,还可以减少数据包时延。在MAC层上实现了该调度方法。仿真结果表明,该方法可以有效地节省能量,并减少了树型拓扑网络的数据包平均时延。
(3)提出了一种WSN分布式进化自学习调度方法。该方法根据WSN中具有相同父节点的近邻节点之间工作特性具有一定相似性的特点,对这些节点的调度策略进行优化。并且针对无线传感器网络节点带宽有限的特点,提出了一种将粒子群优化算法与分布式自学习方法相结合的方法,使节点可以从其它节点学习调度的经验,加快了调度策略的学习速度。仿真结果表明,该方法比占空比10%的S-MAC更节省能量,而数据包时延与占空比60%的S-MAC协议相接近。与分布式自学习调度方法相比较,在仿真结束时,可以进一步节省能耗,并减少数据包平均时延。
Wireless sensor network (WSN) is virtually a resource constrained network system. In most cases, WSN nodes are battery-powered, which restricts the lifetime of WSN. As an embedded system, WSN nodes are also characterized by restrained computation capacity and storage capacity. Moreover, communication bandwidth is much narrower among WSN nodes. Therefore, given these restrictions, it is significantly important to adopt scheduling approaches into the real applications of WSN in order to improve the network performance.
This dissertation conducts an in-depth study of the distributed scheduling approach for wireless sensor network, and proposes four effective distributed scheduling approaches to address the issues of restrained energy. The main contents and contribution of this dissertation are given below:
(1) A family of collaborative distributed scheduling approaches (CDSAs) is proposed to reduce the energy consumption of WSN. The family of CDSAs comprises of two scheduling approaches, i.e. one-step collaborative distributed scheduling approach (O-CDSA) and two-step collaborative distributed scheduling approach (T-CDSA). The family of CDSAs, based on the Markov chain, enables nodes to learn the behavior information of other nodes collaboratively and integrate sleep scheduling with transmission scheduling to reduce the energy consumption. In this dissertation, the adaptability and practicality features of the CDSAs are analyzed and the convergence feature is proved. The test results show that the two proposed approaches can effectively reduce nodes’energy consumption.
(2) A distributed self-learning scheduling approach (SSA) is proposed in order to reduce energy consumption and latency for wireless sensor network. This approach employs reinforcement learning algorithm in scheduling approach of WSN and extends the Q-learning method. Based on a proposed approximate computational method of scheduling parameters, SSA enables nodes to learn continuous transmission parameters and sleep parameter through interacting with the WSN. Then, the WSN nodes can schedule its sleep and transmission through this approach. We implement the SSA in a MAC protocol. The simulation results show that the SSA can effectively reduce energy consumption, and can reduce the average latency of data packs in tree topology network.
(3) A distributed evolutionary self-learning scheduling approach (ESSA) is proposed in order to reduce energy consumption and latency for wireless sensor network. Considering the similarities of working features among the nodes, which have the same parent node and are located close to each other, this approach optimizes their scheduling policy. Given the feature of network bandwidth restriction, ESSA adopts an approach to integrate PSO algorithm and SSA, which enable nodes to profit the experience from other nodes. This approach speeds up the learning rate in great deal. The simulation results further demonstrate that ESSA can largely reduce much more energy than S-MAC with duty cycle 10% while the latency for data packs approximates to that of S-MAC with duty cycle 60%. Comparing with SSA, the ESSA can further reduce the energy consumption and latency of data packs at the completion of the simulation test.
本文对无线传感器网络分布式调度方法进行了深入的研究,并针对无线传感器网络能量受限的特点,提出了四个分布式调度方法。论文的主要贡献和创新点包括:
(1)提出了一类实用的、协作分布式的调度方法。该类方法包含两种调度方法:一步协作分布式调度方法和两步协作分布式调度方法。该类方法基于马尔可夫链,使节点通过相互协作学习其它节点的行为信息,将节点传输调度与休眠调度相结合,达到节省能量的目的。分析了该类方法的适应性、实用性,并从理论上证明了其收敛性。实验结果表明这两个调度方法能够有效地减少能量消耗。
(2)提出了一种WSN分布式自学习调度方法。该方法在WSN分布式调度研究中引入再励学习的思想,通过对Q-学习方法扩展,提出了一种计算调度参数的近似方法,使节点可以得到具有连续值空间的发送调度参数和休眠调度参数,从而使节点实现传输调度和休眠调度。该方法在优化节点能耗的同时,还可以减少数据包时延。在MAC层上实现了该调度方法。仿真结果表明,该方法可以有效地节省能量,并减少了树型拓扑网络的数据包平均时延。
(3)提出了一种WSN分布式进化自学习调度方法。该方法根据WSN中具有相同父节点的近邻节点之间工作特性具有一定相似性的特点,对这些节点的调度策略进行优化。并且针对无线传感器网络节点带宽有限的特点,提出了一种将粒子群优化算法与分布式自学习方法相结合的方法,使节点可以从其它节点学习调度的经验,加快了调度策略的学习速度。仿真结果表明,该方法比占空比10%的S-MAC更节省能量,而数据包时延与占空比60%的S-MAC协议相接近。与分布式自学习调度方法相比较,在仿真结束时,可以进一步节省能耗,并减少数据包平均时延。
Wireless sensor network (WSN) is virtually a resource constrained network system. In most cases, WSN nodes are battery-powered, which restricts the lifetime of WSN. As an embedded system, WSN nodes are also characterized by restrained computation capacity and storage capacity. Moreover, communication bandwidth is much narrower among WSN nodes. Therefore, given these restrictions, it is significantly important to adopt scheduling approaches into the real applications of WSN in order to improve the network performance.
This dissertation conducts an in-depth study of the distributed scheduling approach for wireless sensor network, and proposes four effective distributed scheduling approaches to address the issues of restrained energy. The main contents and contribution of this dissertation are given below:
(1) A family of collaborative distributed scheduling approaches (CDSAs) is proposed to reduce the energy consumption of WSN. The family of CDSAs comprises of two scheduling approaches, i.e. one-step collaborative distributed scheduling approach (O-CDSA) and two-step collaborative distributed scheduling approach (T-CDSA). The family of CDSAs, based on the Markov chain, enables nodes to learn the behavior information of other nodes collaboratively and integrate sleep scheduling with transmission scheduling to reduce the energy consumption. In this dissertation, the adaptability and practicality features of the CDSAs are analyzed and the convergence feature is proved. The test results show that the two proposed approaches can effectively reduce nodes’energy consumption.
(2) A distributed self-learning scheduling approach (SSA) is proposed in order to reduce energy consumption and latency for wireless sensor network. This approach employs reinforcement learning algorithm in scheduling approach of WSN and extends the Q-learning method. Based on a proposed approximate computational method of scheduling parameters, SSA enables nodes to learn continuous transmission parameters and sleep parameter through interacting with the WSN. Then, the WSN nodes can schedule its sleep and transmission through this approach. We implement the SSA in a MAC protocol. The simulation results show that the SSA can effectively reduce energy consumption, and can reduce the average latency of data packs in tree topology network.
(3) A distributed evolutionary self-learning scheduling approach (ESSA) is proposed in order to reduce energy consumption and latency for wireless sensor network. Considering the similarities of working features among the nodes, which have the same parent node and are located close to each other, this approach optimizes their scheduling policy. Given the feature of network bandwidth restriction, ESSA adopts an approach to integrate PSO algorithm and SSA, which enable nodes to profit the experience from other nodes. This approach speeds up the learning rate in great deal. The simulation results further demonstrate that ESSA can largely reduce much more energy than S-MAC with duty cycle 10% while the latency for data packs approximates to that of S-MAC with duty cycle 60%. Comparing with SSA, the ESSA can further reduce the energy consumption and latency of data packs at the completion of the simulation test.
引文
[1] Akyildiz I F, Su W, Sankarasubramaniam Y, Cayirci E. Wireless sensor networks: a survey. Computer Networks, 2002, 38(4):393-422.
[2] Mainwaring A, Culler D, Polastre J, et al. Wireless sensor networks for habitat monitoring. Proceedings of the 1st ACM International Workshop on Wireless Sensor Networks and Applications, Atlanta, GA, USA, September 28, 2002:88–97.
[3] Paek J, Chintalapudi K, Govindan R, et al. Wireless sensor network for structural health monitoring: performance and experience. Proceedings of the Second IEEE Workshop on Embedded Networked Sensors, Sydney, Australia, May 30, 2005:1–10.
[4] Boukerche A, Oliveira H A B, Nakamura E F, et al. Localization systems for wireless sensor networks. IEEE Wireless Communications, 2007, 6:6–12.
[5]任丰原,黄海宁,林闯.无线传感器网络.软件学报, 2003, 14(7):1282-1291.
[6] Roush W, Goho A M, Scigliano E, et al . Ten emerging technologies that will change the world. Technology Review, 2003, 106(1): 33-49.
[7]中华人民共和国科学技术部.国家中长期科学和技术发展规划纲要(2006-2020年). http://www.most.gov.cn/kjgh/.
[8]温家宝.让科技引领中国可持续发展.中国中央人民政府门户网站, 2009-11-23. http://www.gov.cn/ldhd/2009-11/23/content_1471208.htm.
[9]孙利民,李建中,陈渝,等.无线传感器网络.北京:清华大学出版社, 2005.
[10]杨正东,熊庆旭.综合覆盖调度的无线传感器网络跨层路由研究.传感技术学报, 2008, 21(5):814-818.
[11] Kim E-J, Choi H-H. Energy-efficient BOP-based beacon transmission scheduling in wireless sensor networks. IEICE Transactions on Communications, 2008, 11:3469-3479.
[12] Mahfoudh S, Minet P. Survey of energy efficient strategies in wireless ad hoc and sensor networks. In: Proceedings of 7th International Conference on Networking, Mexico: IEEE Computer Society, 2008. 1-7.
[13] Agnihotri S, Nuggehalli P, Jamadagni H S. Correlation, coding, and cooperation in wireless sensor networks. Proceedings of 3rd International Workshop on Algorithmic Aspects of Wireless Sensor Networks (ALGOSENSORS 2007), Wroclaw Poland, Jul 14, 2007; LNCS 4837:83-98.
[14]康波,柯欣,孙利民,等.无线传感器网络中的调度算法研究.计算机科学, 2008, 35(2):47-51.
[15] Keshavarzian A, Lee H, Venkatraman L, et al. Wakeup scheduling in wireless sensor networks. Proc. of MobiHoc’06. New York, USA: ACM, 2006:322-333.
[16] Sharma G, Mazumdar R R, Shroff N B. On the complexity of scheduling in wireless networks. Proc. of MobiCom’06. New York, USA: ACM, 2006: 227-238.
[17] Choi H, Wang J, Hughes EA, Scheduling for information gathering on sensor network. Wireless Networks, 2009, 15(1):127-140.
[18] Chen Y, Fleury E. A distributed policy scheduling for wireless sensor networks. .Proc. of IEEE INFOCOM 2007, Anchorage, AK, 2007:1559-1567.
[19] Wang B. Coverage control in sensor networks, London: Springer, 2010.
[20]文浩,林闯,任丰原,等.无线传感器网络的QoS体系结构.计算机学报, 2009, 32(3):432-440.
[21] Xia F. QoS challenges and opportunities in wireless sensor/actuator networks. Sensors, 2008, 8:1099-1110
[22]孙岩,马华东.无线多媒体传感器网络QoS保障问题.电子学报, 2008, 36(7):1412-1420.
[23]张学,陆桑璐,陈贵海,等.无线传感器网络的拓扑控制.软件学报, 18(4):943-954.
[24]郑国强,李建东,周志立.无线传感器网络MAC协议研究进展.自动化学报, 2008, 34(3):305-316.
[25]任彦,张思东,张宏科.无线传感器网络中覆盖控制理论与算法.软件学报, 2006, 117(3):422-433.
[26]蹇强,龚正虎,朱培栋,等.无线传感器网络MAC协议研究进展.软件学报, 2008, 19(2):389-403.
[27] Yao Y, Giannakis G B. Energy-efficient scheduling for wireless sensor networks. IEEE Transactions on Communications, 2005, 53(8):1333-1342.
[28] Jain S, Srivastava S. A survey and classification of distributed scheduling algorithms for sensor networks. Proc. of SENSORCOM 2007, Washington, DC, USA: IEEE Computer Society, 2007:88-93.
[29] Wang L, Xiao Y. A survey of energy-efficient scheduling mechanisms in sensor networks. Mobile Networks and Applications, 2006. 11(5):723-740.
[30] Tian D, Georganas N D. A node scheduling scheme for energy conservation in large wireless sensor networks. Wireless Communications and Mobile Computing, 2003, 3(2):271– 290.
[31] Xing G L, Wang X, Zhang Y, et al. Integrated Coverage and Connectivity Configuration for Energy Conservation in Sensor Networks. Acm Transactions on Sensor Networks, 2005, 1(1):36-72.
[32] Tian D, Georganas N D. Location and calculation-free node-scheduling schemes in large wireless sensor networks. Ad Hoc Networks, 2004, 2(1), 65-85.
[33] Kumar S, Lai T H, Balogh J. On k-coverage in a mostly sleeping sensor network. Proceedings of the Tenth Annual International Conference on Mobile Computingand Networking (MobiCom’04), Philadelphia, Pennsylvania, USA, 2004:144-158.
[34] Hua C, Yum T P. Asynchronous random sleeping for sensor networks. ACM Transactions on Sensor Networks, 2007, 3(3):15/1-15/25.
[35] Hasegawa G, Matsuo T, Murata M, et al. Comparisons of packet scheduling algorithms for fair service among connections on the Internet. Journal of High Speed Networks, 2003, 12(1-2):1-27.
[36]鞠海玲,崔莉,黄长城. EasiCC:一种保证带宽公平性的传感器网络拥塞控制机制.计算机研究与发展, 2008, 45(1): 16-25.
[37] Yin X, Chen H, Shen Y, et al. A priority-based packet scheduling method in wireless sensor networks. Proc. of IEEE ICIA. 2006:627-631.
[38]孙利民,李波,周新运.无线传感器网络的拥塞控制技术.计算机研究与发展, 2008, 45(1):63-72.
[39] Reggiani L, Spagnolini U. Minimum Interference Distributed Scheduling for Packet Transmission. In: Proc. of IEEE ISSSTA '08. 2008. 411-415.
[40] Chen Y, Zhao Q, Krishnamurthy V, et al. Transmission scheduling for optimizing sensor network lifetime: A stochastic shortest path approach. IEEE Transactions on Signal Processing, 2007. 55(5):2294-2309.
[41] Visweswara S C, Dutta R, Sichitiu ML. Adaptive ad hoc self-organizing scheduling for quasi-periodic sensor network lifetime. Computer Communications, 2006, 29(17):3366-3384.
[42]李方敏,徐文君,刘新华.无线传感器网络功率控制技术.软件学报, 2008, 19(3):716-732.
[43] Bhatia R, Kashyap A, Li L. The power balancing problem in energy constrained multi-hop wireless networks. Proc. of IEEE INFOCOM 2007, 2007:553-561
[44]李方敏,徐文君,高超.一种适用于无线传感器网络的功率控制MAC协议.软件学报, 2007, 18(5):1080-1091.
[45] Pantazis N A, Vergados D J, Vergados D D, et al. Energy efficiency in wireless sensor networks using sleep mode TDMA scheduling. Ad Hoc Networks, 2009, 7(2):322-343.
[46] Lu G, Krishnamachari B, Raghavendra CS. An adaptive energy-efficient and low-latency MAC for data gathering in wireless sensor networks. Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS’04), Santa Fe, NM, United states, 2004:3091-3098.
[47] Gandham S, Dawande M, Prakash R. Link scheduling in wireless sensor networks: distributed edge-coloring revisited. Journal of Parallel and Distributed Computing, 2008, 68(8):1122-1134.
[48] Ye W, Heidemann J, Estrin D. An energy-efficient MAC protocol for wireless sensor networks. Proc. of IEEE INFOCOM 2002. 2002:1567-1576.
[49] Ye W, Heidemann J, Estrin D. Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking, 2004, 12(3):493-506.
[50] Dam T V, Langendoen K. An adaptive energy-efficient MAC protocol for wireless sensor networks. Proceedings of the 1st International Conference on Embedded Networked Sensor System (SenSys’03), Los Angeles, California, USA, 2003:171-180.
[51] Sun Y, Du S, Gurewitz O, et al. DW-MAC: a low latency, energy efficient demand-wakeup MAC protocol for wireless sensor network. Proceedings of the 9th ACM International Symposium on Mobile Ad Hoc Networking And Computing (MobiHoc’08), 2008:53-62.
[52] Du S, Saha A K, Johnson D B. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. Proceedings of the 26th Annual IEEE Conference on Computer Communications (INFOCOM 2007), May 2007:1478-1486.
[53] Sun Y, Gurewitz O, Johnson DB. RI-MAC: A receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems (SenSys’08), Raleigh, NC, USA, 2008:1-14.
[54] Polastre J, Hill J, Culler D. Versatile low power media access for wireless sensor networks. Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems (SenSys’04), Baltimore, MD, USA, 2004:95-107.
[55] Buettner M, Yee G V, Anderson E, et al. X-MAC: A short preamble MAC protocol for duty-cycled wireless sensor networks. Proceedings of the 4th International Conference on Embedded Networked Sensor Systems (SenSys’06), Boulder, Colorado, USA, 2006:307-320.
[56] Rhee I, Warrier A, Aia M, et al. Z-MAC: a Hybrid MAC for wireless sensor networks. IEEE/ACM Transactions on Networking, 2008, 16(3):511-524.
[57] Rhee I, Warrier A C, Xu L. Randomized dining philosophers to TDMA scheduling in wireless sensor networks. Technical report, Computer Science Department, North Carolina State University, Raleigh, NC, 2004.
[58] Yi Y, Veciana G D, Shakkottai S. On optimal MAC scheduling with physical interference. Proc. of IEEE INFOCOM 2007, 2007:294-302.
[59]李建中,高宏.无线传感器网络的研究进展.计算机研究与发展, 2008, 45(1):1-15.
[60] Chatterjea S, Niegerg T, Meratnia N. A distributed and self-organizing scheduling algorithm for energy-efficient data aggregation in wireless sensor networks. ACM Transactions on Sensor Networks, 2008, 4(4):20:1-20:41.
[61] Lin C, He Y X, Peng C, et al. A distributed efficient architecture for wireless sensor networks. Proceedings of 21st International Conference on Advanced InformationNetworking and Applications Workshops-volume 2. Washington, DC, IEEE Computer Society, 2007:429-434.
[62] Badia L, Botta A, Lenzini L. A genetic approach to joint routing and link scheduling for wireless mesh networks. Ad Hoc Networks, 2009, 7(4):654-664.
[63] Fu L, Liew S C, Huang J. Joint power control and link scheduling in wireless networks for throughput optimization. Proc. of ICC 2008. 2008:3066-3072.
[64] Hengstler S. Joint routing, scheduling, and power control in energy-constrained wireless sensor networks. Proceedings of International Conference on Wireless Networks and Emerging Technologies, 2005:190-195.
[65] Sharma G, Shroff N B, Mazumdar R R. Joint congestion control and distributed scheduling for throughput guarantees in wireless networks. Proc. of IEEE INFOCOM 2007, 2007:2072-2080.
[66] Niu J, Deng Z. Collaborative Distributed Scheduling Approaches for Wireless Sensor Network, 2009, 9(10):8007-8030.
[67]牛建军,邓志东.基于马尔可夫链的无线传感器网络分布式调度方法.自动化学报, 2010, 36(5): 685-695.
[68]熊斌斌,林闯,任丰原.无线传感器网络随机投递传输协议性能分析.软件学报. 2009, 20(4):942-953.
[69]柯欣,孙利民.多跳无线传感器网络吞吐量分析.通信学报. 2007, 28(9):78-84.
[70]张衡阳,许丹,刘云辉,等.一种平滑高斯半马尔可夫传感器网络移动模型.软件学报. 2008, 19(7):1707-1715.
[71] Park P, Marco P D, Soldati P, et al. A generalized markov chain model for effective analysis of slotted IEEE 802.15.4. Proceedings of 2009 IEEE 6th International Conference on Mobile Adhoc and Sensor Systems (MASS’09), Macau, China, Oct. 12-15, 2009:130-139.
[72] Martalo M, Busanelli S, Ferrari G. Markov chain-based performance analysis of multihop IEEE 802.15.4 wireless networks. Performance Evaluation, 2009, 66(2009):722-741.
[73] Polin S, Ergen M, Ergen S C, et al. Performance analysis of slotted carrier sense IEEE 802.15.4 medium access layer. IEEE Transactions on Wireless Communications. 2008, 7(9):3359-3371.
[74] Lei J, Yates R, Greenstein L. A generic model for optimizing single-hop transmission policy of replenishable sensors. IEEE Transactions on Wireless Communications. 2009, 8(2):547-551.
[75] Chiasserini C F, Garetto M. An analytical model for wireless sensor networks with sleeping nodes. IEEE Trans. Mob. Comput. 2006, 5:1706-1718.
[76]朱红松,赵磊,徐勇军,等.基于精细梯度策略的多链路协同数据转发协议.软件学报. 2009, 20(11):3045-3059.
[77] Sivrikaya F, Geithner T, Truong C, et al. Stochastic routing in wireless sensor networks. Proceedings of 2009 IEEE International Conference on Communications Workshops (ICC 2009). Dresden, Germany. June 14-18, 2009:598-602.
[78] Hashmi S U, Sarker J H, Mouftah H T, et al. An efficient MAC protocol with correlated connection arrival and variable slot assignment in wireless sensor networks. Proceedings of 2010 IEEE Conference on Communications (ICC 2010), Cape Town, South Africa: IEEE, May 23-27, 2010:1-5.
[79]林恺,赵海,尹震宇,等.无线传感器网络路由中的能量预测及算法实现.通信学报. 2006, 27(5):21-27.
[80] Wang X, Wang S, Ma J, et al. Energy-aware scheduling of surveillance in wireless multimedia sensor networks. Sensors. 2010, 10:3100-3125.
[81] Kang H, Li X, Moran P J. Power-aware markov chain based tracking approach for wireless sensor networks. Proceedings of 2007 IEEE Wireless Communications and Networking Conference (WCNC 2007). Kowloon, China. March 11-15 2007:4212-4217.
[82] Wireless lan medium access control (MAC) and physical layer (PHY) specifications. ANSI/IEEE Std 802.11, 1999 Ed.; IEEE: Hoboken, NJ, USA, June 2003.
[83] Kim S, Fonseca R, Dutta P, et al. Flush: a reliable bulk transport protocol for multihop wireless networks. Proceedings of the 5th ACM Conference on Embedded Networked Sensor Systems, Sydney, Australia, November 6-9, 2007:351-365.
[84] Paek J, Govindan R. RCRT: Rate-controlled reliable transport for wireless sensor networks. Proceedings of the 5th ACM Conference on Embedded Networked Sensor Systems, Sydney, Australia, November 6–9, 2007:305-319.
[85] Geoffrey W A, Stephen D H, Matt W. Lance: optimizing high-resolution signal collection in wireless sensor networks. Proceedings of the 6th ACM Conference on Embedded Networked Sensor Systems, Raleigh, NC, USA, November 5-7, 2008:169-182.
[86]林元烈.应用随机过程.北京:清华大学出版社, 2002.
[87]胡奇英,刘建庸.马尔可夫决策过程引论.西安:西安电子科技大学出版社. 2000.
[88]刘克.实用马尔可夫决策过程.北京:清华大学出版社. 2004.
[89] Crossbow Technology, MPR-MIB Users Manual (Revision A), Crossbow Technology: San Jose, CA, USA, 2007.
[90] Yick J, Mukherjee B, Ghosal D. Wireless sensor network survey. Computer Networks, 2008, 52:2292-2330.
[91] Estrin D. Tutorial wireless sensor networks part IV: sensor network protocols. Presented at the Eighth ACM International Conference on Mobile Computing and Networking (MobiCom 2002), Atlanta, GA, USA, September 23-28, 2002.
[92] Niu J. Self-learning scheduling approach for wireless sensor network Proceedings ofthe 2010 International Conference on Future Computer and Communication (ICFCC 2010), 2010, 3:253-257.
[93] Anastasi G, Conti M, Francesco M D, et al. Energy conservation in wireless sensor networks: a survey. Ad Hoc Networks, 2009, 7(3):537-568.
[94] Cano C, Bellalta B, Sfairopoulou A, et al. A low power listening MAC with scheduled wake up after transmissions for WSNs. IEEE Commun. Lett. 2009, 13:221-223.
[95] Chang Y, Jian J, Sheu J, et al. ADCA: an asynchronous duty cycle adjustment MAC protocol for wireless sensor networks. Proceedings of IEEE GLOBECOM, New Orleans, LA, USA, November, 2008:1-5.
[96] Liu S, Fan K-W, Sinha P. CMAC: an energy-efficient MAC layer protocol using convergent packet forwarding for wireless sensor networks. ACM Transactions on Sensor Networks, 2009, 5(4):29-34.
[97] Sutton R S, Barto A G. Reinforcement Learning: An Introduction. The MIT Press, Cambridge, Massachusetts, London, England, 1998.
[98] Watkins C J C H. Learning from delayed rewards. PhD Thesis, University of Cambridge, England, 1989.
[99] Watkins C J C H. Q-Learning. Machine Learning, 1992, 8:279-292.
[100]高阳,陈世福,陆鑫.强化学习研究综述.自动化学报, 2004, 30(1):86-100.
[101] Gosavi A. Reinforcement learning: a tutorial survey and recent advances. Informas Journal on Computing, 2009, 21(2):178-192.
[102] Bonarini A, Lazaric A, Montrone F, et al. Reinforcement distribution in fuzzy Q-learning, Fuzzy Sets and Systems, 2009, 160(10):1420-1443.
[103] Szepesvari C. Interpolation-based Q-learning. Proceedings of the 21th International Conference on Machine Learning, July 4-8, 2004:791-798.
[104] Hansselt H, Wiering M. Reinforcement learning in continuous action spaces. Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, IEEE Press, April 1-5, 2007:272-279.
[105] Vinceze D, Kovacs S. Fuzzy rule interpolation-based Q-learning. Proceedings of 5th International Symposium on Applied Computational Intelligence and Informatics, Timisoara, Romania, May 28-29, 2009:55-59.
[106] Er M J, Deng C. Online Tuning of fuzzy inference systems using dynamic fuzzy Q-learning. IEEE Transactions on Systems, Man, and Cybernetics- Part B: Cybernetics, 2004, 34(3):1478-1489.
[107] Lazaric A, Restelli M, Bonarini A. Reinforcement learning in continuous action spaces through sequential monte carlo methods. Advances in Neural Information Processing Systems (NIPS), 2007. http://books.nips.cc/nips20.html.
[108] Millan J, Posenato D, Dedieu E. Continuous-Action Q-learning, Machine Learning,2002, 49:247-265.
[109] Meier A, Woehrle M, Zimmerling M, et al. ZeroCal: automatic MAC protocol calibration. Proceedings of the 6th IEEE International Conference on Distributed Computing in Sensor Systems (DCOSS 2010); Lecture Notes in Computer Science, v6131 LNCS, 2010:31-44.
[110] Weiss G. Multiagent Systems: A modern Approach to Distributed Artificial Intelligence. The MIT Press, Cambridge, Massachusetts, London, England, 2000.
[111] Liu Z, Elhanany I. RL-MAC: a reinforcement learning based MAC for wireless sensor networks. International Journal of Sensor Networks, 2006, 1(3-4):117-124.
[112] Du S, Saha A K, Johnson D B. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. Proceedings of the 26th Annual IEEE Conference on Computer Communications (INFOCOM 2007), 2007:1478-1486.
[113] Dutta P, Taneja J, Jeong J, et al. A building block approach to sensornet systems. Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems (Sensys’08), 2008:267-280.
[114] Kennedy J, Eberhart R. Particle swarm optimization. Proceedings of IEEE International Conference on Neural Networks, Washington, 1995:1942-948.
[115] Niu J. Evolutionary Self-learning Scheduling Approach for Wireless Sensor Network. Proceedings of 2010 Third International Conference on Intelligent Computation Technology and Automation (ICICTA2010), Changsha, China, 2010:245-249.
[116]赵志宏,高阳,骆斌,等.多Agent系统中强化学习的研究现状和发展趋势.计算机科学, 2004, 31(3):23-27.
[117] Ikenooue S, Asada M, Hosoda K. Cooperative behavior acquisition by asynchronous policy renewal that enables simultaneous learning in multiagent environment. Proceedings of the 2002 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, EPFL, Lausanne, Switzerland, October 2002:2728-2734.
[118] Tham C, Renaud J. Multi-agent systems on sensor networks: a distributed reinforcement learning approach. Proceedings of the 2005 Intelligent Sensors, Sensor Networks and Information Processing Conference, Melbourne, Australia, December 2005:423-429.
[119]仲宇,顾国昌,张汝波.多智能体系统中的分布式强化学习研究现状.控制理论与应用. 2003, 20(3):317-322.
[120]谢晓锋,张文俊,杨之廉.微粒群算法综述.控制与决策. 2003, 18(2):129-134.
[121] Iima H, Kuroe Y. Swarm reinforcement learning algorithms based on particle swarm optimization. Proceedings of 2008 IEEE International Conference on Systems, Man and Cybernetics (SMC 2008), 2008:1110-1115.
[122] Yang M, Tian Y, Liu X. Cooperative Q-learning based on maturity of the policy. Proceedings of the 2009 IEEE international Conference on Mechatronics andAutomation, Changchun, China, August 9-12, 2009:1352-1356.
[123] Wang Xin, Yang Chunhua, Qin bin. Decouding control using a PSO-based reinforcement learning. Proceedings of 2009 International Conference on Computational Intelligence and natural computing (CINC’09), Wuhan, China: IEEE, June, 2009:170-173.
[124] Iima H, Kuroe Y. Swarm reinforcement learning algorithm based on particle swarm optimization whose personal bests have lifespans. ICONIP 2009, Part II, LNCS 5864, 2009:169-178.
[125] Clerc M. The swarm and the queen: towards a deterministic and adaptive particle swarm optimization. Proceedings of the 1999 Congress on Evolutionary Computation, Washington, DC, 1999:1951-1957.
[126] Heidemann J, Govindan R. An overview of embedded sensor networks, Technical Report ISI-TR-2004-594, USC/Information Sciences Institute, 2004.
[2] Mainwaring A, Culler D, Polastre J, et al. Wireless sensor networks for habitat monitoring. Proceedings of the 1st ACM International Workshop on Wireless Sensor Networks and Applications, Atlanta, GA, USA, September 28, 2002:88–97.
[3] Paek J, Chintalapudi K, Govindan R, et al. Wireless sensor network for structural health monitoring: performance and experience. Proceedings of the Second IEEE Workshop on Embedded Networked Sensors, Sydney, Australia, May 30, 2005:1–10.
[4] Boukerche A, Oliveira H A B, Nakamura E F, et al. Localization systems for wireless sensor networks. IEEE Wireless Communications, 2007, 6:6–12.
[5]任丰原,黄海宁,林闯.无线传感器网络.软件学报, 2003, 14(7):1282-1291.
[6] Roush W, Goho A M, Scigliano E, et al . Ten emerging technologies that will change the world. Technology Review, 2003, 106(1): 33-49.
[7]中华人民共和国科学技术部.国家中长期科学和技术发展规划纲要(2006-2020年). http://www.most.gov.cn/kjgh/.
[8]温家宝.让科技引领中国可持续发展.中国中央人民政府门户网站, 2009-11-23. http://www.gov.cn/ldhd/2009-11/23/content_1471208.htm.
[9]孙利民,李建中,陈渝,等.无线传感器网络.北京:清华大学出版社, 2005.
[10]杨正东,熊庆旭.综合覆盖调度的无线传感器网络跨层路由研究.传感技术学报, 2008, 21(5):814-818.
[11] Kim E-J, Choi H-H. Energy-efficient BOP-based beacon transmission scheduling in wireless sensor networks. IEICE Transactions on Communications, 2008, 11:3469-3479.
[12] Mahfoudh S, Minet P. Survey of energy efficient strategies in wireless ad hoc and sensor networks. In: Proceedings of 7th International Conference on Networking, Mexico: IEEE Computer Society, 2008. 1-7.
[13] Agnihotri S, Nuggehalli P, Jamadagni H S. Correlation, coding, and cooperation in wireless sensor networks. Proceedings of 3rd International Workshop on Algorithmic Aspects of Wireless Sensor Networks (ALGOSENSORS 2007), Wroclaw Poland, Jul 14, 2007; LNCS 4837:83-98.
[14]康波,柯欣,孙利民,等.无线传感器网络中的调度算法研究.计算机科学, 2008, 35(2):47-51.
[15] Keshavarzian A, Lee H, Venkatraman L, et al. Wakeup scheduling in wireless sensor networks. Proc. of MobiHoc’06. New York, USA: ACM, 2006:322-333.
[16] Sharma G, Mazumdar R R, Shroff N B. On the complexity of scheduling in wireless networks. Proc. of MobiCom’06. New York, USA: ACM, 2006: 227-238.
[17] Choi H, Wang J, Hughes EA, Scheduling for information gathering on sensor network. Wireless Networks, 2009, 15(1):127-140.
[18] Chen Y, Fleury E. A distributed policy scheduling for wireless sensor networks. .Proc. of IEEE INFOCOM 2007, Anchorage, AK, 2007:1559-1567.
[19] Wang B. Coverage control in sensor networks, London: Springer, 2010.
[20]文浩,林闯,任丰原,等.无线传感器网络的QoS体系结构.计算机学报, 2009, 32(3):432-440.
[21] Xia F. QoS challenges and opportunities in wireless sensor/actuator networks. Sensors, 2008, 8:1099-1110
[22]孙岩,马华东.无线多媒体传感器网络QoS保障问题.电子学报, 2008, 36(7):1412-1420.
[23]张学,陆桑璐,陈贵海,等.无线传感器网络的拓扑控制.软件学报, 18(4):943-954.
[24]郑国强,李建东,周志立.无线传感器网络MAC协议研究进展.自动化学报, 2008, 34(3):305-316.
[25]任彦,张思东,张宏科.无线传感器网络中覆盖控制理论与算法.软件学报, 2006, 117(3):422-433.
[26]蹇强,龚正虎,朱培栋,等.无线传感器网络MAC协议研究进展.软件学报, 2008, 19(2):389-403.
[27] Yao Y, Giannakis G B. Energy-efficient scheduling for wireless sensor networks. IEEE Transactions on Communications, 2005, 53(8):1333-1342.
[28] Jain S, Srivastava S. A survey and classification of distributed scheduling algorithms for sensor networks. Proc. of SENSORCOM 2007, Washington, DC, USA: IEEE Computer Society, 2007:88-93.
[29] Wang L, Xiao Y. A survey of energy-efficient scheduling mechanisms in sensor networks. Mobile Networks and Applications, 2006. 11(5):723-740.
[30] Tian D, Georganas N D. A node scheduling scheme for energy conservation in large wireless sensor networks. Wireless Communications and Mobile Computing, 2003, 3(2):271– 290.
[31] Xing G L, Wang X, Zhang Y, et al. Integrated Coverage and Connectivity Configuration for Energy Conservation in Sensor Networks. Acm Transactions on Sensor Networks, 2005, 1(1):36-72.
[32] Tian D, Georganas N D. Location and calculation-free node-scheduling schemes in large wireless sensor networks. Ad Hoc Networks, 2004, 2(1), 65-85.
[33] Kumar S, Lai T H, Balogh J. On k-coverage in a mostly sleeping sensor network. Proceedings of the Tenth Annual International Conference on Mobile Computingand Networking (MobiCom’04), Philadelphia, Pennsylvania, USA, 2004:144-158.
[34] Hua C, Yum T P. Asynchronous random sleeping for sensor networks. ACM Transactions on Sensor Networks, 2007, 3(3):15/1-15/25.
[35] Hasegawa G, Matsuo T, Murata M, et al. Comparisons of packet scheduling algorithms for fair service among connections on the Internet. Journal of High Speed Networks, 2003, 12(1-2):1-27.
[36]鞠海玲,崔莉,黄长城. EasiCC:一种保证带宽公平性的传感器网络拥塞控制机制.计算机研究与发展, 2008, 45(1): 16-25.
[37] Yin X, Chen H, Shen Y, et al. A priority-based packet scheduling method in wireless sensor networks. Proc. of IEEE ICIA. 2006:627-631.
[38]孙利民,李波,周新运.无线传感器网络的拥塞控制技术.计算机研究与发展, 2008, 45(1):63-72.
[39] Reggiani L, Spagnolini U. Minimum Interference Distributed Scheduling for Packet Transmission. In: Proc. of IEEE ISSSTA '08. 2008. 411-415.
[40] Chen Y, Zhao Q, Krishnamurthy V, et al. Transmission scheduling for optimizing sensor network lifetime: A stochastic shortest path approach. IEEE Transactions on Signal Processing, 2007. 55(5):2294-2309.
[41] Visweswara S C, Dutta R, Sichitiu ML. Adaptive ad hoc self-organizing scheduling for quasi-periodic sensor network lifetime. Computer Communications, 2006, 29(17):3366-3384.
[42]李方敏,徐文君,刘新华.无线传感器网络功率控制技术.软件学报, 2008, 19(3):716-732.
[43] Bhatia R, Kashyap A, Li L. The power balancing problem in energy constrained multi-hop wireless networks. Proc. of IEEE INFOCOM 2007, 2007:553-561
[44]李方敏,徐文君,高超.一种适用于无线传感器网络的功率控制MAC协议.软件学报, 2007, 18(5):1080-1091.
[45] Pantazis N A, Vergados D J, Vergados D D, et al. Energy efficiency in wireless sensor networks using sleep mode TDMA scheduling. Ad Hoc Networks, 2009, 7(2):322-343.
[46] Lu G, Krishnamachari B, Raghavendra CS. An adaptive energy-efficient and low-latency MAC for data gathering in wireless sensor networks. Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS’04), Santa Fe, NM, United states, 2004:3091-3098.
[47] Gandham S, Dawande M, Prakash R. Link scheduling in wireless sensor networks: distributed edge-coloring revisited. Journal of Parallel and Distributed Computing, 2008, 68(8):1122-1134.
[48] Ye W, Heidemann J, Estrin D. An energy-efficient MAC protocol for wireless sensor networks. Proc. of IEEE INFOCOM 2002. 2002:1567-1576.
[49] Ye W, Heidemann J, Estrin D. Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking, 2004, 12(3):493-506.
[50] Dam T V, Langendoen K. An adaptive energy-efficient MAC protocol for wireless sensor networks. Proceedings of the 1st International Conference on Embedded Networked Sensor System (SenSys’03), Los Angeles, California, USA, 2003:171-180.
[51] Sun Y, Du S, Gurewitz O, et al. DW-MAC: a low latency, energy efficient demand-wakeup MAC protocol for wireless sensor network. Proceedings of the 9th ACM International Symposium on Mobile Ad Hoc Networking And Computing (MobiHoc’08), 2008:53-62.
[52] Du S, Saha A K, Johnson D B. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. Proceedings of the 26th Annual IEEE Conference on Computer Communications (INFOCOM 2007), May 2007:1478-1486.
[53] Sun Y, Gurewitz O, Johnson DB. RI-MAC: A receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems (SenSys’08), Raleigh, NC, USA, 2008:1-14.
[54] Polastre J, Hill J, Culler D. Versatile low power media access for wireless sensor networks. Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems (SenSys’04), Baltimore, MD, USA, 2004:95-107.
[55] Buettner M, Yee G V, Anderson E, et al. X-MAC: A short preamble MAC protocol for duty-cycled wireless sensor networks. Proceedings of the 4th International Conference on Embedded Networked Sensor Systems (SenSys’06), Boulder, Colorado, USA, 2006:307-320.
[56] Rhee I, Warrier A, Aia M, et al. Z-MAC: a Hybrid MAC for wireless sensor networks. IEEE/ACM Transactions on Networking, 2008, 16(3):511-524.
[57] Rhee I, Warrier A C, Xu L. Randomized dining philosophers to TDMA scheduling in wireless sensor networks. Technical report, Computer Science Department, North Carolina State University, Raleigh, NC, 2004.
[58] Yi Y, Veciana G D, Shakkottai S. On optimal MAC scheduling with physical interference. Proc. of IEEE INFOCOM 2007, 2007:294-302.
[59]李建中,高宏.无线传感器网络的研究进展.计算机研究与发展, 2008, 45(1):1-15.
[60] Chatterjea S, Niegerg T, Meratnia N. A distributed and self-organizing scheduling algorithm for energy-efficient data aggregation in wireless sensor networks. ACM Transactions on Sensor Networks, 2008, 4(4):20:1-20:41.
[61] Lin C, He Y X, Peng C, et al. A distributed efficient architecture for wireless sensor networks. Proceedings of 21st International Conference on Advanced InformationNetworking and Applications Workshops-volume 2. Washington, DC, IEEE Computer Society, 2007:429-434.
[62] Badia L, Botta A, Lenzini L. A genetic approach to joint routing and link scheduling for wireless mesh networks. Ad Hoc Networks, 2009, 7(4):654-664.
[63] Fu L, Liew S C, Huang J. Joint power control and link scheduling in wireless networks for throughput optimization. Proc. of ICC 2008. 2008:3066-3072.
[64] Hengstler S. Joint routing, scheduling, and power control in energy-constrained wireless sensor networks. Proceedings of International Conference on Wireless Networks and Emerging Technologies, 2005:190-195.
[65] Sharma G, Shroff N B, Mazumdar R R. Joint congestion control and distributed scheduling for throughput guarantees in wireless networks. Proc. of IEEE INFOCOM 2007, 2007:2072-2080.
[66] Niu J, Deng Z. Collaborative Distributed Scheduling Approaches for Wireless Sensor Network, 2009, 9(10):8007-8030.
[67]牛建军,邓志东.基于马尔可夫链的无线传感器网络分布式调度方法.自动化学报, 2010, 36(5): 685-695.
[68]熊斌斌,林闯,任丰原.无线传感器网络随机投递传输协议性能分析.软件学报. 2009, 20(4):942-953.
[69]柯欣,孙利民.多跳无线传感器网络吞吐量分析.通信学报. 2007, 28(9):78-84.
[70]张衡阳,许丹,刘云辉,等.一种平滑高斯半马尔可夫传感器网络移动模型.软件学报. 2008, 19(7):1707-1715.
[71] Park P, Marco P D, Soldati P, et al. A generalized markov chain model for effective analysis of slotted IEEE 802.15.4. Proceedings of 2009 IEEE 6th International Conference on Mobile Adhoc and Sensor Systems (MASS’09), Macau, China, Oct. 12-15, 2009:130-139.
[72] Martalo M, Busanelli S, Ferrari G. Markov chain-based performance analysis of multihop IEEE 802.15.4 wireless networks. Performance Evaluation, 2009, 66(2009):722-741.
[73] Polin S, Ergen M, Ergen S C, et al. Performance analysis of slotted carrier sense IEEE 802.15.4 medium access layer. IEEE Transactions on Wireless Communications. 2008, 7(9):3359-3371.
[74] Lei J, Yates R, Greenstein L. A generic model for optimizing single-hop transmission policy of replenishable sensors. IEEE Transactions on Wireless Communications. 2009, 8(2):547-551.
[75] Chiasserini C F, Garetto M. An analytical model for wireless sensor networks with sleeping nodes. IEEE Trans. Mob. Comput. 2006, 5:1706-1718.
[76]朱红松,赵磊,徐勇军,等.基于精细梯度策略的多链路协同数据转发协议.软件学报. 2009, 20(11):3045-3059.
[77] Sivrikaya F, Geithner T, Truong C, et al. Stochastic routing in wireless sensor networks. Proceedings of 2009 IEEE International Conference on Communications Workshops (ICC 2009). Dresden, Germany. June 14-18, 2009:598-602.
[78] Hashmi S U, Sarker J H, Mouftah H T, et al. An efficient MAC protocol with correlated connection arrival and variable slot assignment in wireless sensor networks. Proceedings of 2010 IEEE Conference on Communications (ICC 2010), Cape Town, South Africa: IEEE, May 23-27, 2010:1-5.
[79]林恺,赵海,尹震宇,等.无线传感器网络路由中的能量预测及算法实现.通信学报. 2006, 27(5):21-27.
[80] Wang X, Wang S, Ma J, et al. Energy-aware scheduling of surveillance in wireless multimedia sensor networks. Sensors. 2010, 10:3100-3125.
[81] Kang H, Li X, Moran P J. Power-aware markov chain based tracking approach for wireless sensor networks. Proceedings of 2007 IEEE Wireless Communications and Networking Conference (WCNC 2007). Kowloon, China. March 11-15 2007:4212-4217.
[82] Wireless lan medium access control (MAC) and physical layer (PHY) specifications. ANSI/IEEE Std 802.11, 1999 Ed.; IEEE: Hoboken, NJ, USA, June 2003.
[83] Kim S, Fonseca R, Dutta P, et al. Flush: a reliable bulk transport protocol for multihop wireless networks. Proceedings of the 5th ACM Conference on Embedded Networked Sensor Systems, Sydney, Australia, November 6-9, 2007:351-365.
[84] Paek J, Govindan R. RCRT: Rate-controlled reliable transport for wireless sensor networks. Proceedings of the 5th ACM Conference on Embedded Networked Sensor Systems, Sydney, Australia, November 6–9, 2007:305-319.
[85] Geoffrey W A, Stephen D H, Matt W. Lance: optimizing high-resolution signal collection in wireless sensor networks. Proceedings of the 6th ACM Conference on Embedded Networked Sensor Systems, Raleigh, NC, USA, November 5-7, 2008:169-182.
[86]林元烈.应用随机过程.北京:清华大学出版社, 2002.
[87]胡奇英,刘建庸.马尔可夫决策过程引论.西安:西安电子科技大学出版社. 2000.
[88]刘克.实用马尔可夫决策过程.北京:清华大学出版社. 2004.
[89] Crossbow Technology, MPR-MIB Users Manual (Revision A), Crossbow Technology: San Jose, CA, USA, 2007.
[90] Yick J, Mukherjee B, Ghosal D. Wireless sensor network survey. Computer Networks, 2008, 52:2292-2330.
[91] Estrin D. Tutorial wireless sensor networks part IV: sensor network protocols. Presented at the Eighth ACM International Conference on Mobile Computing and Networking (MobiCom 2002), Atlanta, GA, USA, September 23-28, 2002.
[92] Niu J. Self-learning scheduling approach for wireless sensor network Proceedings ofthe 2010 International Conference on Future Computer and Communication (ICFCC 2010), 2010, 3:253-257.
[93] Anastasi G, Conti M, Francesco M D, et al. Energy conservation in wireless sensor networks: a survey. Ad Hoc Networks, 2009, 7(3):537-568.
[94] Cano C, Bellalta B, Sfairopoulou A, et al. A low power listening MAC with scheduled wake up after transmissions for WSNs. IEEE Commun. Lett. 2009, 13:221-223.
[95] Chang Y, Jian J, Sheu J, et al. ADCA: an asynchronous duty cycle adjustment MAC protocol for wireless sensor networks. Proceedings of IEEE GLOBECOM, New Orleans, LA, USA, November, 2008:1-5.
[96] Liu S, Fan K-W, Sinha P. CMAC: an energy-efficient MAC layer protocol using convergent packet forwarding for wireless sensor networks. ACM Transactions on Sensor Networks, 2009, 5(4):29-34.
[97] Sutton R S, Barto A G. Reinforcement Learning: An Introduction. The MIT Press, Cambridge, Massachusetts, London, England, 1998.
[98] Watkins C J C H. Learning from delayed rewards. PhD Thesis, University of Cambridge, England, 1989.
[99] Watkins C J C H. Q-Learning. Machine Learning, 1992, 8:279-292.
[100]高阳,陈世福,陆鑫.强化学习研究综述.自动化学报, 2004, 30(1):86-100.
[101] Gosavi A. Reinforcement learning: a tutorial survey and recent advances. Informas Journal on Computing, 2009, 21(2):178-192.
[102] Bonarini A, Lazaric A, Montrone F, et al. Reinforcement distribution in fuzzy Q-learning, Fuzzy Sets and Systems, 2009, 160(10):1420-1443.
[103] Szepesvari C. Interpolation-based Q-learning. Proceedings of the 21th International Conference on Machine Learning, July 4-8, 2004:791-798.
[104] Hansselt H, Wiering M. Reinforcement learning in continuous action spaces. Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, IEEE Press, April 1-5, 2007:272-279.
[105] Vinceze D, Kovacs S. Fuzzy rule interpolation-based Q-learning. Proceedings of 5th International Symposium on Applied Computational Intelligence and Informatics, Timisoara, Romania, May 28-29, 2009:55-59.
[106] Er M J, Deng C. Online Tuning of fuzzy inference systems using dynamic fuzzy Q-learning. IEEE Transactions on Systems, Man, and Cybernetics- Part B: Cybernetics, 2004, 34(3):1478-1489.
[107] Lazaric A, Restelli M, Bonarini A. Reinforcement learning in continuous action spaces through sequential monte carlo methods. Advances in Neural Information Processing Systems (NIPS), 2007. http://books.nips.cc/nips20.html.
[108] Millan J, Posenato D, Dedieu E. Continuous-Action Q-learning, Machine Learning,2002, 49:247-265.
[109] Meier A, Woehrle M, Zimmerling M, et al. ZeroCal: automatic MAC protocol calibration. Proceedings of the 6th IEEE International Conference on Distributed Computing in Sensor Systems (DCOSS 2010); Lecture Notes in Computer Science, v6131 LNCS, 2010:31-44.
[110] Weiss G. Multiagent Systems: A modern Approach to Distributed Artificial Intelligence. The MIT Press, Cambridge, Massachusetts, London, England, 2000.
[111] Liu Z, Elhanany I. RL-MAC: a reinforcement learning based MAC for wireless sensor networks. International Journal of Sensor Networks, 2006, 1(3-4):117-124.
[112] Du S, Saha A K, Johnson D B. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. Proceedings of the 26th Annual IEEE Conference on Computer Communications (INFOCOM 2007), 2007:1478-1486.
[113] Dutta P, Taneja J, Jeong J, et al. A building block approach to sensornet systems. Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems (Sensys’08), 2008:267-280.
[114] Kennedy J, Eberhart R. Particle swarm optimization. Proceedings of IEEE International Conference on Neural Networks, Washington, 1995:1942-948.
[115] Niu J. Evolutionary Self-learning Scheduling Approach for Wireless Sensor Network. Proceedings of 2010 Third International Conference on Intelligent Computation Technology and Automation (ICICTA2010), Changsha, China, 2010:245-249.
[116]赵志宏,高阳,骆斌,等.多Agent系统中强化学习的研究现状和发展趋势.计算机科学, 2004, 31(3):23-27.
[117] Ikenooue S, Asada M, Hosoda K. Cooperative behavior acquisition by asynchronous policy renewal that enables simultaneous learning in multiagent environment. Proceedings of the 2002 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, EPFL, Lausanne, Switzerland, October 2002:2728-2734.
[118] Tham C, Renaud J. Multi-agent systems on sensor networks: a distributed reinforcement learning approach. Proceedings of the 2005 Intelligent Sensors, Sensor Networks and Information Processing Conference, Melbourne, Australia, December 2005:423-429.
[119]仲宇,顾国昌,张汝波.多智能体系统中的分布式强化学习研究现状.控制理论与应用. 2003, 20(3):317-322.
[120]谢晓锋,张文俊,杨之廉.微粒群算法综述.控制与决策. 2003, 18(2):129-134.
[121] Iima H, Kuroe Y. Swarm reinforcement learning algorithms based on particle swarm optimization. Proceedings of 2008 IEEE International Conference on Systems, Man and Cybernetics (SMC 2008), 2008:1110-1115.
[122] Yang M, Tian Y, Liu X. Cooperative Q-learning based on maturity of the policy. Proceedings of the 2009 IEEE international Conference on Mechatronics andAutomation, Changchun, China, August 9-12, 2009:1352-1356.
[123] Wang Xin, Yang Chunhua, Qin bin. Decouding control using a PSO-based reinforcement learning. Proceedings of 2009 International Conference on Computational Intelligence and natural computing (CINC’09), Wuhan, China: IEEE, June, 2009:170-173.
[124] Iima H, Kuroe Y. Swarm reinforcement learning algorithm based on particle swarm optimization whose personal bests have lifespans. ICONIP 2009, Part II, LNCS 5864, 2009:169-178.
[125] Clerc M. The swarm and the queen: towards a deterministic and adaptive particle swarm optimization. Proceedings of the 1999 Congress on Evolutionary Computation, Washington, DC, 1999:1951-1957.
[126] Heidemann J, Govindan R. An overview of embedded sensor networks, Technical Report ISI-TR-2004-594, USC/Information Sciences Institute, 2004.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |