Concurrent VSAM access for batch and CICS

Understanding VSAM SHAREOPTIONs

SHAREOPTIONs are settings that provide various levels of shared access to VSAM files. This capability has been available for decades, and on initial review, it appears to be an easy way to provide file sharing. Let's take a closer look at the SHAREOPTIONs and exactly what it takes to implement this approach.

SHAREOPTION 1 is the most restrictive of all SHAREOPTIONs, disallowing any file sharing if updates occur. This SHAREOPTION permits a single region to write to a VSAM data set or enables many regions to read a data set. If a region is updating the file, all access to the file is denied to any region seeking to perform a read, so this option is often deemed unacceptable. Yet, performance wise, this is the most effective SHAREOPTION to use.

SHAREOPTION 2 also prohibits concurrent data set updates, but provides for multiple reads during an update process. This function is typically used where CICS is the updating application with infrequent data reading from batch. A major drawback of SHAREOPTION 2 occurs when a reading region attempts to access data that an update region has modified. It is possible that the reading region will not receive the most recent updates and could terminate abnormally if substantial data changes have occurred.

SHAREOPTION 3 on the surface, appears to solve the limitations of SHAREOPTION 2 by enabling multiple, concurrent updates. Because SHAREOPTION 3 permits multiple regions to both read and write to a single data set, it would also permit multiple regions to concurrently update identical blocks of records, causing a loss of physical data. Using SHAREOPTION 3 could lead to VSAM data integrity issues and increased overhead.

SHAREOPTION 4 requires extensive and complex programming modifications to your application. These modifications will increase I/O overhead to ensure data integrity. Use of SHAREOPTION 4 will not enable a single CICS region to lock onto its required records. An environment using SHAREOPTION 4 and enabling concurrent updates by both CICS and batch could experience a severe data integrity problem in the event of a CICS transaction abend. The CICS dynamic transaction backout will replace all the transaction-updated records with initial values, even if a batch job updated the records.

Practical implications: Employment of a higher VSAM SHAREOPTION translates into increased overhead. Additionally, the risk of error increases when there is less data-integrity protection. Employing VSAM SHAREOPTIONs for concurrent update access between CICS regions and batch jobs is rarely a suitable solution for companies striving to ensure processing efficiency and data integrity.

File open and close products

For many data centers, an option for managing UPDATE access to VSAM data from multiple address spaces includes disabling CICS files, or closing them, for a period of time in order to perform batch processing. Closing files to CICS while batch is running has been around since the late 1970s. For companies that are not facing pressure to reduce or even eliminate the nightly batch window, closing files to CICS is still a good solution. It minimizes the disruption to users by taking selected files offline, as batch needs them. This solution is also a low-cost way to protect data integrity.

Practical Implications: When files are closed, users do not have full access to their applications. Multiple batch jobs that need to process against a given file must be serialized because only one address space can own a VSAM file at a given time. This further lengthens the time the file is offline to CICS. As a result, productivity is lost, customer service stops, and work flow is interrupted. In a world that requires 24/7 availability, taking files offline to CICS has become far less acceptable.

Shadow files

Another method for improving VSAM availability is the use of shadow or mirror files. This procedure involves closing the VSAM master file to CICS while a backup copy (the shadow or mirror file) is taken. Once this is complete, the shadow or mirror file is opened to CICS in READ ONLY mode. The batch process then updates the master file. When the batch processing finishes, the updated master file is then reallocated to CICS with full READ/WRITE access.

The drawback of a shadow file is that it only remains a true duplicate for a short time. Once batch begins its updating process, the files are no longer considered mirrored. In addition, the CICS user community is limited to inquiry only and this impacts productivity. While the shadow file approach has typically been used to respond to ad hoc or scheduled report requests, it still requires that either a CICS or batch process wait until the files are reallocated and available. Again, multiple batch jobs that need to process against a given file must be serialized because only one address space can own a VSAM file at a given time. This will further lengthen the time the file is offline to CICS.

Using shadow files is an affordable solution to preserve the integrity of data. However, this solution can be costly to end users because of lost time and outdated data. Furthermore, you cannot run batch processes during the day or night while CICS remains available with full READ/WRITE access.

Practical Implications: A shadow file remains a true duplicate for a short period of time. Once the batch processing begins, the files are no longer considered mirrored. Read-only mode impacts user productivity and precludes CICS file access. Additionally, a batch cannot run while CICS has full READ/WRITE access.

Memo posting

Memo posting takes the concept of shadow files one step further. With memo posting, CICS access to a file is set to READ ONLY while batch updates the master copy of the file. [You can do this by switching the CICS File Control Table definition to READ ONLY and using SHAREOPTION (2,3) to allow CICS to read the VSAM master file and give exclusive UPDATE access to the batch address space.] The CICS application architecture needs to be designed to recognize this condition. Update requests that the CICS application makes to the VSAM file are queued in a log so that they can be applied later. When the batch process finishes, the queued CICS updates are then applied to the master file and the file is restored to CICS with full READ/WRITE access.

A point of consideration is that the CICS transaction is reading records through its buffering facilities while batch is directly (natively) accessing and updating the master VSAM file. This can present data integrity exposures that must be considered and managed. For example, records that reside in the CICS buffers might not reflect the updates the batch job is processing natively against the master VSAM file. When this occurs, a CICS transaction will get a dirty read and will make an update decision based on that dirty read. From the batch perspective, the batch program might natively read a master VSAM record for update, but CICS might have already read that same record and queued its own update. This situation causes the batch program to get a dirty read.

With memo posting, you must carefully manage data integrity. If you do not have an application that handles this situation, you have to code your application to handle it, assuming you have the ability to change source code. As with the practices of opening and closing files and shadow files, memo posting is not a true real-time solution for providing up-to-date data, nor does it allow batch to run while CICS remains available with concurrent READ/WRITE access. As previously noted, multiple batch jobs that need to process against a given file must be serialized.

Practical Implications: Memo posting is not a true real-time solution for providing up-to-date data and it creates data integrity exposures that must be carefully considered and managed.

EXCI

EXCI, or external CICS interface, is an application programming interface (API) that allows non-CICS resources to gain access to CICS applications running in a CICS Transaction Server region. With EXCI, these non-CICS resources and CICS applications can also pass and receive data using a communications area (commarea). The EXCI API is intended to allow the non-CICS resources to tap into CICS sporadically to complete occasional updates. Another common use of EXCI is to open and close files. However, using EXCI for batch jobs that process numerous records significantly increases the time it takes for the job to finish. Due to these performance issues with high-volume jobs, the use of EXCI for VSAM file sharing is limited.

Similar to the other methods already discussed, EXCI requires extensive and complex coding changes. If you are doing a large numbers of updates, you need to consider frequently syncpointing or committing the work. If you do not use syncpoints, CICS will see the entire batch job as a single unit of work and it will retain locks on all updated records for recoverable VSAM files until the batch job finishes. If you do commit the work along the way, you need to determine a strategy to manage recovery if a batch abend occurs. Going back to a point-in-time copy of the file and restoring it will undo the batch updates, but what happens to the CICS updates to those files?

The requirement for making application changes makes EXCI an unacceptable choice for companies that use third-party applications whose code cannot be changed or other applications whose code they do not want changed. Even organizations that have access to the application source and are open to redesigning it typically find that the data integrity exposure and performance impact limit the use of this solution.

Practical Implications: Due to performance issues with high-volume jobs, EXCI is not suitable for VSAM file sharing. It also requires extensive and complex coding changes that are not possible or desirable with many applications. Using EXCI for batch jobs that process numerous records significantly increases the time it takes for the job to finish. Due to these performance issues with high-volume jobs, the use of EXCI for VSAM file sharing is limited.

VSAM RLS

Prior to the introduction of VSAM RLS, a single CICS address space owned and accessed any VSAM data sets opened for update. The CICS address space processed all requests for data. VSAM RLS provides a new method for accessing VSAM files that gives multiple CICS address spaces concurrent READ/WRITE access to recoverable VSAM data sets. This approach eliminates the potential problem of the one CICS address space that owned UPDATE access to the VSAM files from being a single point-of-failure or a performance bottleneck.

From a batch perspective, VSAM RLS allows a batch program to read recoverable VSAM files with integrity. VSAM RLS does this by employing an SMSVSAM address space and a coupling facility. These two pieces provide record locking and buffering for the multiple CICS address spaces that can access the VSAM files for READ or WRITE. For recoverable VSAM files, batch receives READ access through these buffers to ensure data integrity. However, to make batch updates to recoverable files and to protect data integrity, organizations still must rely on the file open and close process, shadow files, or memo posting. If you were to choose memo posting, you would need to do some work with CEDA and switch CICS to a non-RLS, read-only file control table (FCT). This is because you cannot open an RLS FCT if you have the file open for update in batch.

From a CICS and batch perspective, you can have concurrent UPDATE access for CICS and batch, but only if the VSAM files are specified as nonrecoverable. This means that VSAM RLS will not be responsible for data integrity. For example, if a batch abend occurs, you can restore the file to a point-in-time and recover from the batch updates, but you will lose all CICS updates after the batch job failed.

Transactional VSAM

The main objective of transactional VSAM, also known as TVS or DFSMStvs, is to extend the ability to share recoverable VSAM files between CICS and batch for update without compromising data integrity. From a concurrent update standpoint, SMSVSAM and the coupling facility manage all record locks for both CICS and batch, just as they do for VSAM RLS. From a recovery standpoint, TVS (in conjunction with resource recovery services or RRS) manages syncpointing and recovery for non-CICS VSAM updates and CICS provides recovery for online updates.

TVS requires you to use VSAM RLS and make extensive modifications to the batch application, such as committing work with syncpoints and managing recovery considerations. This effort is similar to the one you must make for EXCI. If you undertake this application redesign, IBM recommends making your batch applications restartable from the point at which the last unit of work was committed. All of these changes can, and usually do, represent a major redesign and rewrite of the batch applications.

Sharing VSAM files inside of CICS: A way to break free from previous limitations

One way to avoid the aforementioned conflicts is to share VSAM files inside of CICS. This makes batch jobs appear to CICS like any other online transaction. Batch jobs can then process while CICS continues to have full READ/WRITE access to VSAM files. Because CICS has already been entrusted with your data, extending CICS to batch is a natural move. There are numerous advantages to this approach including: performance, data integrity, implementation, reliability, and recovery benefits.

Like other solutions, you must consider issues associated with solutions that share files inside of CICS. These issues include performance, such as how you will make batch and CICS each perform optimally and ensure CICS service level agreements (SLAs) are not impacted. You also need to look at recovery. How do you recover quickly while maintaining the integrity of CICS updates, batch updates, or both?

The optimal transparent VSAM file sharing solution should address these areas while retaining the investment that you have made in your batch applications.