Flash储存管理的虚拟盘系统设计与实现
|
摘要
本人实习所在的公司是一家专注于移动通信和多媒体终端的Soc设计的高科技外商独资企业。公司掌握多项有关移动通信和多媒体方面的核心技术,有多名在硅谷十年以上资深的IC设计专家提供芯片设计方案;积累了从算法仿真、系统设计、协议、软件平台及生产测试工具的开发,直至客户化终端的多方面成熟的行业和专业经验,为广大生产厂家提供了多种解决方案,赢得了较高的声誉。公司产品主要涉及GSM/GPRS终端手机,CD/MP3/WMA播放器。同时,公司的产品线还覆盖了WCDMA/TD-SCDMA手机以及多媒体手机等。
本课题来源于公司实际开发的一个项目:CSW Platform,这个项目是开发一个GSM/GPRS通讯软件平台,其主要应用于移动电话及AT模块。本人所在的开发小组在项目中负责的是文件系统模块及对于Flash存储的抽象化管理。随着手机上的Flash存储器容量的扩大和要处理的对象(音频,视频,图片,铃声等)的增加,实现嵌入式文件系统是一种必然的趋势。文件系统内部又划分为上层的VFS(Virtual File System)及下层的FAT。该文件系统实现了支持Microsoft的FAT12/16/32文件系统,并可挂载多个不同类型的文件系统及存储设备。Flash存储器具有不同于磁盘的硬件特性,而我们设计的Flash储存管理系统则借鉴了虚拟内存管理及Java的垃圾回收机制,通过对各种不同类型的Flash存储器进行抽象和封装,从而让文件系统(或者其他应用)能够像访问硬盘那样访问Flash存储器,并保证了数据的安全及完整性。
CSW Platform的最初设计是以Flash为存储介质的Application(SMS,Reg,FS等)都直接操作Flash,这意味着这些Application都需要考虑flash的硬件特性。这带来了两个方面的问题:工作重复(每个Application都要完成类似的工作),从而导致维护成本相对较高;实现难度加大,稳定性降低(实现和Flash的硬件特性相关)。
为解决这两个问题,我们提出虚拟磁盘系统(VDS,Virtual Disk System)方案。VDS是在Application(SMS,Reg,FS等)和Flash存储介质之间引入的一个抽象层,这层界面向上层用户屏蔽Flash存储器的物理特性,当上层Application传来写数据命令时,VDS为Application提供了称为Virtual Disk的存储介质访问接口。换言之,对Application来说,Vritual Disk和真正的Hard Disk没有区别(或者说无法区分):
(1)整个Virtual Disk被划分为多个block,Application根据block的index来访问block;多个连续的block构成一个分区(partition);
(2)Application访问block的最小单位是block;
(3)在同一个block上可以重复写。
VDS方案将对存储介质的管理和对Application本身的对象的管理分割开来,屏蔽了Flash设备的硬件特性,提高了整个CSW Platform的模块性;易于移植基于Hard Disk的Application到我们的系统上。同时这样还能对Flash存储空间充分的利用,能保证Flash区块的平均磨损,延长Flash设备的使用寿命。
本文将详细阐述关于如何管理Flash存储的虚拟磁盘系统(VDS,Virtual DiskSystem)的设计与实现。在这个虚拟磁盘系统中,我们为上层用户提供了一个虚拟磁盘,上层用户可以不考虑Flash的特性,而是在我们提供的虚拟磁盘上运行其操作及数据的存储,从而大大提高了CSW Plattform的兼容性。所有的VDS的应用接口我们是采用C语言进行编码,并且在论文中将详细描述接口的流程,以及关于垃圾回收机制、断电保护机制的实现。
I internships where the company is a focus on mobile communication and multimedia terminal Soc design of high-tech foreign-owned enterprises. Companies have a number of the mobile communications and multimedia aspects of the core technology, a number more than 10 years in Silicon Valley senior IC design experts to provide chip design; accumulated from the algorithm simulation, system design, protocol, software platform and production test tools The development, until the end of the multifaceted, mature industries and professional experience, for the majority of the manufacturer provides a wide range of solutions, has won high reputation. Company products are mainly related to GSM / GPRS mobile terminals, CD/MP3/WMA player. At the same time, the company's product line also covers the WCDMA / TD-SCDMA mobile phones and multimedia mobile phones.
This topic comes from the actual development of a project: CSW Platform, this project is to develop a GSM / GPRS communications software platform, which are mainly used in mobile phones and AT module. I of the development team in the project is responsible for the file system for the Flash memory module and the abstract management. With the phone's Flash memory and the expansion of capacity to deal with the object (audio, video, pictures, ringtones, etc.) increase, and embedded file system is an inevitable trend. Documents within the system and into the upper VFS (Virtual File System) and the lower deck of the FAT. The file system has the support of Microsoft FAT12/16/32 file system, and to mount a number of different types of file systems and storage devices. Flash memory is different from the disk hardware features, and we design the Flash storage management system will draw on the virtual memory management and Java's garbage collection mechanism, through all the different types of Flash memory to the abstract and package, so that File system (or other applications) can be as hard as access to Flash memory, and to ensure the safety and integrity of the data.
CSW Platform design is the first Flash storage medium for the Application (SMS, Reg, FS, etc.) have a direct operation Flash, which means that they need to consider the Application flash hardware features. This has brought the two areas: duplication of work (each Application must be completed similar work), resulting in relatively high maintenance costs; difficult to achieve stability, lower (Flash and the realization of related hardware).
To address these two issues, our virtual disk system (VDS, Virtual Disk System) programme. VDS is in the Application (SMS, Reg, FS) and Flash storage medium between the introduction of a level of abstraction, this upper layer user interface to shield the physical characteristics of Flash memory, when the upper Application came to write data order, VDS for the Application The provision known as the Virtual Disk storage media access interface. In other words, the Application for, Vritual Disk and really no difference between the Hard Disk (or not distinguish between):
(1) the Virtual Disk is divided into several block, Application according to the index to block access block; multiple consecutive constitute a sub-block (partition);
(2) Application to block the smallest unit of the block;
(3) in the same block, can repeat write.
VDS programmes on the management and storage medium for the Application of the object itself separated from the management, shielding the Flash equipment hardware features, enhance the entire CSW Platform module; easy to transplant based on the Application Hard Disk to our system. At the same time so can the Flash storage space fully utilized, can guarantee that the average wear block Flash, Flash to extend the useful life of equipment.
In this paper, I will elaborate on how to manage the Flash memory of virtual disk system (VDS) Design and Implementation. In this virtual disk system, we have the upper provides a virtual disk, the upper Flash users can not consider the characteristics, but we provide a virtual disk on the operation of its operation and data storage, thus greatly improving the compatibility of the CSW Plattform Sexual. All the VDS application interface is used coded language to C, and in the paper I will describe in detail the interface process, and on garbage collection mechanism, power protection mechanisms to achieve.
本课题来源于公司实际开发的一个项目:CSW Platform,这个项目是开发一个GSM/GPRS通讯软件平台,其主要应用于移动电话及AT模块。本人所在的开发小组在项目中负责的是文件系统模块及对于Flash存储的抽象化管理。随着手机上的Flash存储器容量的扩大和要处理的对象(音频,视频,图片,铃声等)的增加,实现嵌入式文件系统是一种必然的趋势。文件系统内部又划分为上层的VFS(Virtual File System)及下层的FAT。该文件系统实现了支持Microsoft的FAT12/16/32文件系统,并可挂载多个不同类型的文件系统及存储设备。Flash存储器具有不同于磁盘的硬件特性,而我们设计的Flash储存管理系统则借鉴了虚拟内存管理及Java的垃圾回收机制,通过对各种不同类型的Flash存储器进行抽象和封装,从而让文件系统(或者其他应用)能够像访问硬盘那样访问Flash存储器,并保证了数据的安全及完整性。
CSW Platform的最初设计是以Flash为存储介质的Application(SMS,Reg,FS等)都直接操作Flash,这意味着这些Application都需要考虑flash的硬件特性。这带来了两个方面的问题:工作重复(每个Application都要完成类似的工作),从而导致维护成本相对较高;实现难度加大,稳定性降低(实现和Flash的硬件特性相关)。
为解决这两个问题,我们提出虚拟磁盘系统(VDS,Virtual Disk System)方案。VDS是在Application(SMS,Reg,FS等)和Flash存储介质之间引入的一个抽象层,这层界面向上层用户屏蔽Flash存储器的物理特性,当上层Application传来写数据命令时,VDS为Application提供了称为Virtual Disk的存储介质访问接口。换言之,对Application来说,Vritual Disk和真正的Hard Disk没有区别(或者说无法区分):
(1)整个Virtual Disk被划分为多个block,Application根据block的index来访问block;多个连续的block构成一个分区(partition);
(2)Application访问block的最小单位是block;
(3)在同一个block上可以重复写。
VDS方案将对存储介质的管理和对Application本身的对象的管理分割开来,屏蔽了Flash设备的硬件特性,提高了整个CSW Platform的模块性;易于移植基于Hard Disk的Application到我们的系统上。同时这样还能对Flash存储空间充分的利用,能保证Flash区块的平均磨损,延长Flash设备的使用寿命。
本文将详细阐述关于如何管理Flash存储的虚拟磁盘系统(VDS,Virtual DiskSystem)的设计与实现。在这个虚拟磁盘系统中,我们为上层用户提供了一个虚拟磁盘,上层用户可以不考虑Flash的特性,而是在我们提供的虚拟磁盘上运行其操作及数据的存储,从而大大提高了CSW Plattform的兼容性。所有的VDS的应用接口我们是采用C语言进行编码,并且在论文中将详细描述接口的流程,以及关于垃圾回收机制、断电保护机制的实现。
I internships where the company is a focus on mobile communication and multimedia terminal Soc design of high-tech foreign-owned enterprises. Companies have a number of the mobile communications and multimedia aspects of the core technology, a number more than 10 years in Silicon Valley senior IC design experts to provide chip design; accumulated from the algorithm simulation, system design, protocol, software platform and production test tools The development, until the end of the multifaceted, mature industries and professional experience, for the majority of the manufacturer provides a wide range of solutions, has won high reputation. Company products are mainly related to GSM / GPRS mobile terminals, CD/MP3/WMA player. At the same time, the company's product line also covers the WCDMA / TD-SCDMA mobile phones and multimedia mobile phones.
This topic comes from the actual development of a project: CSW Platform, this project is to develop a GSM / GPRS communications software platform, which are mainly used in mobile phones and AT module. I of the development team in the project is responsible for the file system for the Flash memory module and the abstract management. With the phone's Flash memory and the expansion of capacity to deal with the object (audio, video, pictures, ringtones, etc.) increase, and embedded file system is an inevitable trend. Documents within the system and into the upper VFS (Virtual File System) and the lower deck of the FAT. The file system has the support of Microsoft FAT12/16/32 file system, and to mount a number of different types of file systems and storage devices. Flash memory is different from the disk hardware features, and we design the Flash storage management system will draw on the virtual memory management and Java's garbage collection mechanism, through all the different types of Flash memory to the abstract and package, so that File system (or other applications) can be as hard as access to Flash memory, and to ensure the safety and integrity of the data.
CSW Platform design is the first Flash storage medium for the Application (SMS, Reg, FS, etc.) have a direct operation Flash, which means that they need to consider the Application flash hardware features. This has brought the two areas: duplication of work (each Application must be completed similar work), resulting in relatively high maintenance costs; difficult to achieve stability, lower (Flash and the realization of related hardware).
To address these two issues, our virtual disk system (VDS, Virtual Disk System) programme. VDS is in the Application (SMS, Reg, FS) and Flash storage medium between the introduction of a level of abstraction, this upper layer user interface to shield the physical characteristics of Flash memory, when the upper Application came to write data order, VDS for the Application The provision known as the Virtual Disk storage media access interface. In other words, the Application for, Vritual Disk and really no difference between the Hard Disk (or not distinguish between):
(1) the Virtual Disk is divided into several block, Application according to the index to block access block; multiple consecutive constitute a sub-block (partition);
(2) Application to block the smallest unit of the block;
(3) in the same block, can repeat write.
VDS programmes on the management and storage medium for the Application of the object itself separated from the management, shielding the Flash equipment hardware features, enhance the entire CSW Platform module; easy to transplant based on the Application Hard Disk to our system. At the same time so can the Flash storage space fully utilized, can guarantee that the average wear block Flash, Flash to extend the useful life of equipment.
In this paper, I will elaborate on how to manage the Flash memory of virtual disk system (VDS) Design and Implementation. In this virtual disk system, we have the upper provides a virtual disk, the upper Flash users can not consider the characteristics, but we provide a virtual disk on the operation of its operation and data storage, thus greatly improving the compatibility of the CSW Plattform Sexual. All the VDS application interface is used coded language to C, and in the paper I will describe in detail the interface process, and on garbage collection mechanism, power protection mechanisms to achieve.
引文
[1]Algorithms and Data Structures for Flash Memories,ERAN GAL AND SIVAN TOLEDO,Tel-Aviv University,ACM Computing Surveys,Vol.37,No.2,June 2005,pp.138-163.
[2]The Logical Disk:A New Approach to Improving File Systems,Wiebren de Jonge Dept.of Mathematics and Computer Science,Vrije Universiteit,Amsterdam,M.Frans Kaashoek and Wilson C.Hsieh Laboratory for Computer Science,MIT,Cambridge,MA
[3]LFS File Manager Software:LFM,SAMUEL DUFOUR,MEMORY COMPONENTS DIVISION,DEBORAH SEE,SENIOR SOFTWARE ENGINEER,October 1995
[4]OMG.Unified Modeling Language,version 1.4.http://ww.omg.org:OMG.2001.9
[5]Douglass B.P.Rel-time UML.U.S.A.:Addison-Wesley,1998
[6]Booch G.Object-oriented Anlysis and Design with Applications.U.S.A.:Benjamin/Cummings,1999
[7]OMG.UML Profile for Schedulability,Performance,and Time,version 1.0.http://www.omg.org:OMG.2002.3
[8]李力.闪速存储器技术现状及其发展趋势.单片机与嵌入式系统应用,2001(9)
[9]陈峰.嵌入式系统中的Flash存储管理.单片机与嵌入式系统应用,2003920
[2]The Logical Disk:A New Approach to Improving File Systems,Wiebren de Jonge Dept.of Mathematics and Computer Science,Vrije Universiteit,Amsterdam,M.Frans Kaashoek and Wilson C.Hsieh Laboratory for Computer Science,MIT,Cambridge,MA
[3]LFS File Manager Software:LFM,SAMUEL DUFOUR,MEMORY COMPONENTS DIVISION,DEBORAH SEE,SENIOR SOFTWARE ENGINEER,October 1995
[4]OMG.Unified Modeling Language,version 1.4.http://ww.omg.org:OMG.2001.9
[5]Douglass B.P.Rel-time UML.U.S.A.:Addison-Wesley,1998
[6]Booch G.Object-oriented Anlysis and Design with Applications.U.S.A.:Benjamin/Cummings,1999
[7]OMG.UML Profile for Schedulability,Performance,and Time,version 1.0.http://www.omg.org:OMG.2002.3
[8]李力.闪速存储器技术现状及其发展趋势.单片机与嵌入式系统应用,2001(9)
[9]陈峰.嵌入式系统中的Flash存储管理.单片机与嵌入式系统应用,2003920