READERS ARE CAUTIONED THAT THIS DOCUMENT MAY CONTAIN SOURCE CODE IN THE FORM OF PATCHES OR CODING EXAMPLES.
DESPITE USE OF THE BEST AVAILABLE OPTICAL CHARACTER RECOGNITION EQUIPMENT, AND VISUAL COMPARISON OF THE RESULTS WITH THE ORIGINAL AFTERWARD, THE INTEGRITY AND CORRECTNESS OF ANY CODE SEGMENTS IN THIS DIGITIZED DOCUMENT CANNOT BE GUARANTEED.
AS WAS TRUE IN THE ORIGINAL, NO RESPONSIBILITY IS ASSUMED BY ANYONE IF YOU CHOOSE TO USE THE MATERIAL IN THIS DOCUMENT.
______
THE STANDARD TRUETYPE FONTS “ARIAL,” “TIMES NEW ROMAN,” AND “COURIER NEW” HAVE BEEN USED WHERE POSSIBLE. BE AWARE THAT FONT SUBSTITUTION MAY CAUSE THE RESULTING DOCUMENT TO BE MISALIGNED.
THE ORIGINAL PUBLICATION OF THIS DOCUMENT WAS IN:
Proceedings of the Digital Equipment Computer Users Society, Los Angeles, California - December 1981, pp. 691 - 695.
THE ORIGINAL OF THIS DOCUMENT WAS PART OF THE COLLECTION OF JIM BOSTWICK, LAST CHAIRMAN OF THE U.S. DECUS RSX SPECIAL INTEREST GROUP, AND WAS DONATED BY HIS ESTATE. "Requiem aeternam dona eis, Domine, et lux perpetua luceat eis."
THIS DOCUMENT WAS DIGITIZED AND CONVERTED TO MS WORD FORMAT BY MACHINE INTELLIGENCE. OUR BUSINESS IS RSX, PDP-11, VAX, VMS AND DEC SOFTWARE, SYSTEMS ANALYSIS AND CONSULTING.
VISIT OUR WEB SITE AT FOR MORE RSX AND DEC INFORMATION.
Digitized from the original 24 November 2004 by Machine Intelligence, RSX Software and Consulting -
What Is A Virtual Disk?
Ralph Stamerjohn
Monsanto
St. Louis, MO
ABSTRACT
Virtual disks are files that RSX-11M thinks are real disks. This paper discusses how the virtual disk package was conceived, various ways virtual disks can the used, and potential problems with virtual disks. Finally, it discusses possible extensions to the package.
A virtual disk is a file. Fortunately, RSX11M is easily fooled into thinking that a file is a disk. I started thinking about files as disks in late 1978 when Monsanto added a RP06 disk drive to its RSX11M based program development system. Before that time, the system ran from three RK05s. Project development proceeded serially because only one project at a time could use the one removable drive. With the RP06, we had lots of disk space and everyone could work on their projects at once. However, this also caused some problems:
1.All of our production systems were RK05 based. We still needed to generate RK05 distribution kits and we still had only one removable drive.
2.Everyone had their files organized for their own RK05 disk packs. Programmers used the same UICs for different reasons, various versions of system libraries from [1,1], and their own source maintenance and backup standards.
3.Programmers tend to use any available resource to its capacity. It would not take long to chew up the RP06, and RSX11M has no mechanism for setting or enforcing disk quotas on one's peers or bosses.
Like any good organization, we had a meeting to discuss these problems. At one point during the arguments, someone joked that we could solve all our problems by getting Digital to sell a single RP06 that looked to the system like 70 RK05s. Later at the local pub, I started thinking out loud about the problem. The conversation went something like this (fortunately, my fellow programmers are used to humoring me):
"What is a disk?"
"Huh?"
"A disk is just a bunch of blocks, right?"
"Drink your beer, Ralph."
"A file is just a bunch of blocks too, right?"
"Someone call Esther, he can't drive home like this."
"Look, a disk is just a set of contiguous blocks. So, if I allocate a contiguous file, is that a disk?"
"No, Ralph."
"Yes it is. Look, the disk drivers work with real block numbers, right? So if somehow, I had a new device which took the block numbers, added the start of a contiguous file to them, and passed the request onto the real driver, I could make a file into a disk."
"Huh?"
Two days later, that is exactly what happened. I wrote a device driver which had a database that looked exactly like a disk. The driver's only purpose was to add the starting physical block number of a contiguous file to the block number in the IO.RLB or IO.WLB packet and then pass the request on to the real disk driver. For example, assume DB0:[7,114]GARBAGE.DSK was a contiguous file starting at block number 72345. When MOU tried to read the home block of GARBAGE.DSK (block 1), the virtual disk driver added 72345 to the I/O packet and passed it on to the DB: disk driver. The driver actually read block 72346, but MOU did not care, it only wanted a home block.
1.0 IMPLEMENTATION
Virtual disks are made up of two parts: The contiguous disk file and the virtual disk driver. A virtual disk is completely transparent to RSX11M; that is, it appears to be a unique and real device, separate from the disk on which the file resides. You must INItialize it, MOUnt the disk for use, create UFDs, and run VFY occasionally. The only things you cannot do to a virtual disk are hardware or software boot it or SAVe an image on it. The actual implementation of the virtual disk software involves four components:
VDTBLVirtual disk driver database
VDDRVVirtual disk driver
AVDAssign file to a virtual disk
DVDDeassign file from a virtual disk
1.1 VDTBL Details
VDTBL implements a device data structure (DCB, UCB, and SCB) for a device named VDxx:. The number of units generated is defined by the symbol V$$D11. The data structures are exact copies of the structures RSX11M uses for disks with some specific exceptions:
1.The SCB vector address (S.VCT) and CSR address (S.CSR) are set to null values.
2.RSX11M uses UCB characteristics words 2 and 3 (U.CW2 and U.CW3) to hold the size of the disk in blocks. For virtual disks these words are initially zero, and set by AVD to the size of the file.
3.When initially loaded, VDDRV marks all UCBs as offline (US.OFL in U.ST2). This prevents any I/O to a virtual disk until brought online with AVD. AVD clears this bit when a file is successfully assigned to a virtual disk unit. When the file is deassigned with DVD, the offline bit is set again.
4.An additional word is added in the UCB after U.VCB. This word points to the control structure used to map the disk file currently assigned to the UCB. A nonzero value means the virtual disk unit is assigned to a file. The control structure is created in system Pool by AVD and destroyed by DVD. The structure has the following format:
X.OFF(2 words) Starting physical block number of the file. This location holds the value VDDRV adds to the I/O packet block number to relocate the read or write request to the correct block inside the virtual disk file.
X.UCB(1 word) UCB address of the physical disk data structure. This UCB address is used to get the actual disk controller SCB to which VDDRV queues the I/O packet for service (i.e., the disk drive on which the file is located).
X.FLG(1 word) Flag word. Currently only one bit (XF.RON=1) is used. This bit means that only read access is allowed to the virtual disk file.
X.FNB(15 words) Disk file filename block. This is the FCS filename block of the virtual disk file. It is used by DVD to find the file when the virtual disk is deassigned.
1.2 VDDRV Details
VDDRV is the trivial part of the package. All it does is get the I/O packet from its queue ($GTPKT), check that the block number is legal for the file size ($BLKCK), perform the readonly check if XF.RON is set, add the starting physical block number of the file to the I/O packet block number, and queue the I/O packet to the actual disk driver.
1.3 AVD Details
AVD's purpose is to set up the data structures defined in VDTBL so VDDRV works. It is a privileged RSX11M program and mapped to the Executive. Its basic flow is as follows:
1.Get and parse a command line. Legal commands are shown below:
AVD VDnn:=filespec[/sw]
/CR:nCreate contiguous file of size 'n' blocks
/RKCreate contiguous file of size 4800 blocks
/ROSet up read-only access to file
1.Parse the virtual disk device (VDnn:) and check to see that specific unit exists and the driver is loaded.
2.Parse the file name and check for syntax errors.
3.Check that the specified virtual disk device (VDnn:) is not already assigned.
4.Allocate the extension area from system Pool.
5.If necessary, (/CR or /RK), create a new file. This is done using Files11 QIOs instead of FCS so that contiguous files larger that 65,767 blocks can be created. Note: The files are protected [RW,RW,,], primarily so they cannot be deleted inadvertently.
6.Access the file, get its size and starting physical block number, mark it as locked, and deaccess it. The lock bit is set so that while the file is assigned to a virtual disk, someone else cannot do something to it unintentionally. Obviously, PIP could be used to unlock the file while it is assigned. The statistics block in the file header is read to check if the file is contiguous, and if it is, to get the file size and starting physical block number.
7.Fill in the extension area with the necessary information and store its address in the virtual disk unit's UCB. Also store the file size in U.CW2 and U.CW3 of the UCB.
8.Finally, mark the virtual disk drive on-line and exit.
1.4 DVD Details
DVD reverses what is done by AVD. Again, it is a privileged RSX11M program and mapped to the Executive. Its basic flow is as follows:
1.Get and parse a command line. DVD accepts command lines in the following format:
DVD VDnn:
2.Parse the virtual disk device (VDnn:) and test that the specific unit exists and the driver is loaded.
3.Check that the specified virtual disk device (VDnn:) is assigned and not mounted. DVD requires the disk be dismounted first before it will terminate the assignment. Otherwise, the effect would be the same as removing a mounted RK05 from a disk drive and inserting another pack - instant disaster.
4.Mark the drive off-line so no further accesses can be made.
5.Using the filename block information in the extension area, unlock the file.
6.Deallocate the extension area from system Pool and zero the pointer in the UCB.
2.0 APPLICATIONS
The main use of virtual disks is to package files. This can be applied in many ways. Consider the following:
1.If your disk drives are large enough (RP series), virtual disks can be used to hold Digital's distribution kits. System generations can then be performed on-line and you are guaranteed to not disturb the running system. This is especially useful when generating systems for different CPUs. By using one virtual disk to hold the distribution kit and another disk as the target, a completely set up, packaged disk image can be made for transfer to another CPU.
2.Another very useful application is keeping different software projects completely separate from each other. By using a separate virtual disk for each project, you no longer have to worry about UIC and filename conflicts. This also allows each project to maintain separate [1,1] areas if this is needed for different library support.
3.Virtual disks can be used as a primitive RSX-11M disk quota system. By giving each user a virtual disk file of some appropriate size and requiring all of their files to be stored on that virtual disk, you can limit the amount of disk space they use.
4.Virtual disks make the problem of generating disk kits of different sizes trivial. Our original application, still used today, is to generate RK05 kits on the RP06. By using 4800 block virtual disks, we set up exact RK05 images for transfer either to an RK05 or magtape using DSC or BRU.
5.Virtual disks provide an extra measure of protection because the files and directories in it do not show up in directories of the full disk. This allows you to have a private volume of your own files while actually working on a public disk. In addition, other user's SY:[*,*]*.*.;*/DE do not affect you because the virtual disk files do not have delete access.
Virtual disks can also be used in more exotic ways: Partitioning extremely large non-Digital disks into sizes Digital software can handle, to provide RT11 format disks for RT11 backup and usage, and as a technique to aid dualport access to a single disk spindle.
3.0 PROBLEMS AND CONSIDERATIONS
Now for the bad news. There are problems with virtual disks and some weaknesses you should consider before using them. Among the more serious problems:
1.AVD relies on F11ACP to be correct when it reports the file is contiguous. If somehow F11ACP is fooled into reporting the file as contiguous when it is not, complete disaster can occur. This happened to me when BRU restored a virtual disk file noncontiguous, but marked it as contiguous. Use of the file wiped out the actual disk's index file! A solution would be for AVD to read the file header(s) and check to make sure the file is contiguous.
2.The large contiguous files used by the package are troublesome to back up. Traditionally, Digital utilities like DSC and BRU do not work correctly on multiheader, contiguous files. It took us 2 years to finally get BRU for RSX11M V3.2 patched enough to work with our 120,000 block virtual disk files.
3.AVD is privileged, and as such, file protection by F11ACP is ignored when AVD accesses a file. AVD should perform the protection check so users are limited by the normal file protection rules.
4.AVD and DVD do not perform a privilege check on the terminal, so if installed, any user can create, assign, or deassign a disk.
5.AVD does not mark the virtual disk file as accessed. This allows the physical disk to be dismounted and removed while virtual disks are still on-line.
6.VDTBL assumes that all disks are NPR/DMA devices and sets UC.NPR in the UCB. This would cause transfers to fail if virtual disks were created on a nonDMA device like a RX01.
There are also caveats to using virtual disks. Virtual disks waste some actual disk space. The free space inside a virtual disk file is only available to users of the virtual disk, and not for general allocation from the physical disk.
If a virtual disk file is wasting too much space, it is fairly easy to correct. DSC and BRU will copy from one virtual disk to another. The virtual disk files may be different sizes has long as the output disk file is large enough to hold the allocated space on the input disk file. Therefore, if you have a 50,000 block virtual disk file and find that only 20,000 blocks are really needed, create a 20,000 block virtual disk file and DSC or BRU to it. You can then delete the 50,000 block file to recover the space on the physical disk pack.
Virtual disk files also present a problem for backups and restores. For example, if BRU is used to save the entire physical disk, it will copy the virtual disk files as entire files, no matter how much space inside the virtual disk file is used. Also, individual files inside the virtual disk cannot be selectively restored. The problem is increased for incremental backups. AVD updates the virtual disk file revision date whenever it assigns the disk to a virtual disk unit. Unless the disks are excluded from an incremental backup, BRU will save the entire virtual disk file when using the BRU /REVISED switch, even if no files inside the virtual disk file were changed. At Monsanto, our backup procedure excludes [*,*]*.DSK;*. These files are then assigned individually to VD7:, backed up to a separate saveset, and then deassigned. This still presents a problem because BRU will only let the first saveset on a tape cross to another tape.
Finally, virtual disks cause overhead to the system. VDDRV's processing time is about 0.3 milliseconds per I/O request. More importantly, the package allows you to create many disks where before there was only one. This means more system Pool will be used for F11ACP volume control blocks, index file window headers, and the virtual disk extension blocks. However, there can also be some performance improvements. Specifically, because fewer UICs need to be created on the physical disk, directory searches may be much faster. Also, if you have the memory to burn, you can use separate ACPs for each virtual disk and get some performance improvements by having multiple F11ACP I/O requests in progress.
4.0 IMPROVEMENTS
There are three enhancements which have been made to my original submission on the Spring 1979 RSX SIG tapes (New Orleans). Eric Levy in [301,046] on the Fall 1980 RSX SIG tapes (San Diego) has a version of AVD named AVDX. This version takes a physical block number and size for input instead of a filename. This allows extremely large disks to be easily partitioned into normal Digital disks, i.e., RP06.
Mike Fraser in [370,020] on the Spring 1981 RSX SIG tapes (Miami) has modified AVD to allow file placement by disk block number on creation, creation of RX01size virtual disk files, and a listing option to output the current file assignment to a virtual disk unit. He has also updated VDDRV and VDTBL for RSX11MPlus.