Introduction: The SPD engine SPD (Scalable Performance Data) is designed for high-performance data transfer, which can achieve fast data transfer is due to specific formatted data to maximize the multi-CPU implementation parallel input / output function. The SPD engine uses multi-threaded parallel quick read data, but the corresponding OS must support the allowable thread on any CPU, although this function is the most important function of SPDE, but this function is from SAS data. Support for stored structures, this data organization way to capacity to process the ability of the I / 0 task, but it is not possible to replace the default SAS Base engine, SPDE provides a high-speed processing of large data. Select, especially when reading and writing millions of records, and some OS is not allowed to be greater than 2GB file existence, it is mainly manifested in the following: supporting GB data has measurable test on multiple CPU machines Sexual Where Process Parallel Load Parallel INDEX Creating Parallel Data Transport Inherent Sort by SPDE SPDE can be run in UNIX, Windows, Z / OS (on ZFS file system) and OPENVMS Alpha (only on ODS-5 file system) on
Using the SMP (SYMMETRIC MULTIPROCESSING) Machine SPDE technology is fully built on the SMP machine, one SMP machine has multiple CPUs and OS support threads, in general, it has multiple controllers and multiple disk devices, each disk device Corresponding to a controller, when the SPDE read data is, it runs one or more threads on each CPU, and these threads read data from multiple disk devices, this reading is likely to pass multi-CPU controlled For example: a perfect system, there are 5 CPUs and 10 disk devices, and the SMP machine can also be supported in addition to multi-thread I / 0 when performing data read. Multi-threaded application, such as the SORT process
How to use the SPDE organization SAS data because SPDE is handled for high performance data, the physical storage of the dataset is different from the data store of Base SAS, and the latter is stored in a single file, which contains data and data. The relevant description information, and the SPDE is a separate file used to store data and data description information, and if the data set is indexed, two index files will be generated for each index, these four file types The SPDE component file, each type of file has its own file extension, and each component file can be composed of one or more physical files, so it can span multiple volumes but are used as a reference. Logical files, such as: SPDE can establish many physical files, but can be used as a single file descriptive data component as a single file descriptive data component is different from the actual data component: 1. You can specify a fixed length partition size for the actual data component, specify with the Partsize option, however, you cannot control the partition size of the descriptive data component. The data component file can be built on a different path, but the descriptive data can only be built in a single path until the storage space of the path is full, and the next path metadata component file will be used as .mdf, usually A SPDE dataset has only one MDF file index component file. If a file is indexed, the SPDE will generate two index component files for each index, the HBX extension file contains the global index, and the .idx file contains points The index data component file of the segment is composed of several or more files, better than a single file, each file is fixed, this length is specified in the design, the size of the partition size It will affect the performance performance of your application, because each file is read by a thread's comparison Base SAS engine and SPD engine, and 2 is in SAS database storage data; SAS libraries are one containing multiple files And catalog, however, things under the SPDE library can exist under different paths, you can use the libName syntax to specify some of the details of some data stores containing data files, metadata files, index files, pay attention to The SPDE library is not supported by directory, view, MDDB, or other files.
The SPDE File Operations Space SPDE File Action Space is an additional space required for SPDE operations. If you do not specify default, but when you operate large data volume, it is not enough space. Therefore, you can specify the SPDEUTILLOC option to specify the SPDE file operation space temporary data set stored by the TEMP option, the following libName User SPDE '/ mydata' temp = yes; data a; x = 1; Run; proc print data = a; Note: The use of the user option allows only one level name when referenced) 2 other excessive differences [Feature] [SPD Engine] [Base Sas Engine]] YES NO Parallel Where Optimizes YES NO Minimum Lock Level Member Level Recording level Different requests Alternative to the data set read-read-write support SAS / Share no yes SAS BY automatic sorting YES no directory support No Yes view Support No Yes MDDB Support NO YES Integrity Constrained No Yes Variable Multiple 32767 The number of 63 times of the record 2 is 32 (32-bit machines) The two engine sets of data sets of the data set must be converted to the SPDE data set to be accessed by SPDE, you can be easy. Use COPY, APPEND Procedure, or Data Step (Except for Proc Migrate) In addition, the SAS program you already exist can run in the SPDE engine, just do a small modification, except for the libName syntax sharing SPDE file SPDE support member level Lock, Meaning Multiple users can read the same SPDE dataset at the same time, however, other users can only access it when a data set is updated, if you want to support read and write, then you need SPD server products
