Reputation: 1

Concatenating files with similarDSN

please how can I make a JCL to concatenate various datasets with similar names but differing last qualifiers? We have the recording date of each one as the last qualifier of the DSN. My problem is I don't know when it will be executed and I can't write the date of each file.

Example: AAAAA.BBBBB.CCCCC.F090622

      AAAAA.BBBBB.CCCCC.F100622

      AAAAA.BBBBB.CCCCC.F110622

My question is: IS THERE A WAY TO WRITE JUST ONE DSN AAAA.BBBB.CCCC.F* in my JCL, to concatenate all DSN describe above?

Upvotes: 0

Answers (2)

phunsoft

Reputation: 2745

Use Case for Generation Data Groups

I understand that the requirement is as follows:

Job A runs daily creating a new instance of data set X.
Job B runs periodically, say monthly, processing the content of all instances of data set X created since last run.
You no longer need the individual instances of data set X after job B successfully ran.
You don't need to access a single instance of data set X by date (very often).

If the assumptions are correct, using Generation Data Groups might be an alternative to using data set names with a date qualifier.

See z/OS DFSMS Access Method Services Commands, z/OS MVS JCL User's Guide, and z/OS MVS JCL Reference for details.

Generation Data Groups Explained Briefly

A Generation Data Group, short GDG, is a group of data sets, called Generation Data Sets, short GDS, that are referred by their position in the group. This position is specified by a relative position, enclosed in parenthesis, and appended to the data set name when specifying the DSN in JCL. (You cannot use this notation in TSO/ISPF).

The GDG is a special entry in the data set catalog. It defines the base name for the GDSs, the maximum number of generations that may exist at any time, what happens to the oldest generation, when that maximum is reached, and yet another new GDS is created, plus some more options.

The system keeps track of GDSs as they are created and deleted, and this book keeping is done with the help of the GDG base entry, and an additional qualifier that is appended to the DSN automatially. This qualifier is of form GnnnnV00, where nnnn is the generation number of the instance, incremented by one for each new GDS.

New GDSs are created in JCL by specifying the DSN as specified in the GDG base, and appending the relative generation number as a positive integer. The first new GDS in a job has relative position +1, the second new GDS in the same job has relative position +2. The latest GDS that existed before the job started to execute has relative position zero (0), the one before that has postition -1.

Say, you define a GDG with name Z08593.GDGTEST. You would then create a new GDS instance in JCL as follows:

//OUTPUT DD DSN=Z08593.GDGTEST(+1),DISP=(NEW,CATLG),....

and you would refer to this instance with position (+1) throughout the job. If you need to read the latest instance that existed before the job started, you would refer to it as:

//INPUT DD DSN=Z08593.GDGTEST(0),DISP=OLD,....

As second new instance created within the same job is referred to by DSN=Z08593.GDGTEST(+2). The second but latest instance is referred to by DSN=Z08593.GDGTEST(-1), and so on. By default, the system takes kinda "snapshot" of existing generations just before a job is started, and all relative references throughout the job's execution remaind fixed towards this snapshot.

There is more behind GDGs, and GDSs, and how to handle them. Carefully read the JCL User's Guide (see above) to get a better idea of this, and think about handling exceptions like ABENDs, dulicate runs of a job (because there was a problem with it), etc.

Reading All Existing Generations as One Data Set (Contatenation)

And here is the reason why GDGs migth matter for you: You can easily concatenate all current instances, or generations, of a GDG with a single DD statement in JCL. simply specify the DSN but don't refer to any relative generation:

//INPUT DD DSN=Z08593.GDGTEST,DISP=SHR

The system will create a DD concatenation for you behind the scene. You can even specify in which order the data sets shall be concatenated: LIFO or FIFO:

//INPUT DD DSN=Z08593.GDGTEST,DISP=SHR,GDGORDER=FIFO
    
//INPUT DD DSN=Z08593.GDGTEST,DISP=SHR,GDGORDER=LIFO

with LIFO being the default.

Sample JCLs and JOBLOGs

An example and the resulting job logs help to get the idea, I hope.

Creating the Generation Data Group

This is a one time job. Once the GDG entry has been created, it will exist until deleted explicitly.

//jobname  JOB  ...
//STEP01   EXEC PGM=IDCAMS                             
//SYSPRINT DD  SYSOUT=*                                
//SYSIN    DD  *                                       
  DEFINE GDG ( -                                       
    NAME( Z08593.GDGTEST ) -                           
    LIMIT( 35 ) -                                      
    NOEMPTY -                                          
    SCRATCH                                            
    )                                                  
/*

This job defines a GDG called Z08593.GDGTEST, and specifies that there may be no more than 35 generation data set instances at any time. It also specifies that the oldest generation will be deleted (SCRATCH) when a 36th generation will be created. NOEMPTY means that only the single oldest GDS will be deleted. See the Access Method Services Commands manual for details.

The resulting GDG entry can be inspected. e.g. in ISPF 3.4 by entering the command LISTC ENT(/) ALL on the line of the DGD. The result looks similar to this:

GDG BASE ------ Z08593.GDGTEST                                                                    
     IN-CAT --- CATALOG.ZOS6                                                                      
     HISTORY                                                                                      
       DATASET-OWNER-----(NULL)     CREATION--------2022.167                                      
       RELEASE----------------2     LAST ALTER------2022.167                                      
     ATTRIBUTES                                                                                   
       LIMIT-----------------35      NOSCRATCH  NOEMPTY    LIFO       NOPURGE    NOEXTENDED       
     ASSOCIATIONS--------(NULL)

Creating a New Generation Data Set

This job will run periodically, maybe daily or weekly, and will create a new GDS instance every time it runs.

//jobname  JOB  ...
//STEP01   EXEC PGM=IEBGENER                                    
//SYSPRINT DD   SYSOUT=*                                        
//SYSIN    DD   DUMMY                                           
//SYSUT1   DD   *
Record on GDG data set 01
/*                                                              
//SYSUT2   DD   DSN=Z08593.GDGTEST(+1),                         
//         DISP=(NEW,CATLG,DELETE),                             
//         SPACE=(1,(50,50),RLSE),AVGREC=K

In reality, the program would read a different input each time the job runs, process it, and write the result to the new GDS. For testing purposes, I'm using IEBGENER which simply copies input (//SYSUT1) to output (//SYSUT2). You might want to edit the "record", e.g. ba incrementing the number for each run.

In the log of the first run of this job, one can see that the new GDS's name:

IGD101I SMS ALLOCATED TO DDNAME (SYSUT2  )                 
        DSN (Z08593.GDGTEST.G0001V00                     ) 
        STORCLAS (ZXPS) MGMTCLAS (        ) DATACLAS (ZXPD)
        VOL SER NOS= ZXPL01

In the second run, it looks like this:

IGD101I SMS ALLOCATED TO DDNAME (SYSUT2  ) 
        DSN (Z08593.GDGTEST.G0002V00                     )
        STORCLAS (ZXPS) MGMTCLAS (        ) DATACLAS (ZXPD)
        VOL SER NOS= ZXPL01

and so on. After three runs, you would see the following in ISPF 3.4:

DSLIST - Data Sets Matching Z08593.GDG*                             Row 1 of 4  
Command ===>                                                  Scroll ===> PAGE  
                                                                                
Command - Enter "/" to select action                  Message           Volume  
------------------------------------------------------------------------------- 
         Z08593.GDGTEST                                                 ??????  
         Z08593.GDGTEST.G0001V00                                        ZXPL01  
         Z08593.GDGTEST.G0002V00                                        ZXPL01  
         Z08593.GDGTEST.G0003V00                                        ZXPL01

As you can see, the system adds a generation qualifier as described above to each data set. Note while you cannot work with the relative generation syntax in ISPF, you can always work with the full data set name as shown here. It's just not so useful if you need to often work with those in ISPF: You cannot easily say which generation was created on what date.

Processing all Existing Generations at Once

So now it is time to run the job to process all the generations that had been create since last time this job was run. Here is the JCL for this job:

//jobname  JOB  ...
//STEP01   EXEC PGM=IEBGENER                  
//SYSPRINT DD   SYSOUT=*                      
//SYSIN    DD   DUMMY                         
//SYSUT1   DD   DISP=(SHR,KEEP),                     
//             DSN=Z08593.GDGTEST,            
//             GDGORDER=FIFO                  
//SYSUT2   DD   SYSOUT=*

You specify a single DD statement referring to the GDG by its base name. The system will expand this to a DD concatenation, one DD per generation that exist at the time the job is started. In the job log you can easily confirm this:

IEF142I Z08593R STEP01 - STEP WAS EXECUTED - COND CODE 0000   
IEF285I   Z08593.Z08593R.JOB09672.D0000101.?           SYSOUT 
IGD104I Z08593.GDGTEST.G0001V00                      RETAINED,  DDNAME=SYSUT1
IGD104I Z08593.GDGTEST.G0002V00                      RETAINED,  DDNAME=      
IGD104I Z08593.GDGTEST.G0003V00                      RETAINED,  DDNAME=

Resetting the GDG after Successful Processing all Generations

Once the processing job has successfully worked on all the generations, you want to delete all of them to start the next cycle. Again, an easy job that looks like this:

//jobname  JOB  ...
//STEP01   EXEC PGM=IEFBR14
//GDG      DD   DISP=(OLD,DELETE),                     
//             DSN=Z08593.GDGTEST

The relevant part of the job log shows the data sets have been deleted:

IEF142I Z08593D STEP01 - STEP WAS EXECUTED - COND CODE 0000                     
IGD105I Z08593.GDGTEST.G0001V00                      DELETED,   DDNAME=SYSUT1   
IGD105I Z08593.GDGTEST.G0002V00                      DELETED,   DDNAME=         
IGD105I Z08593.GDGTEST.G0003V00                      DELETED,   DDNAME=

Note it is important to understand that only the GDSs are deleted, the GDG base is not.

Upvotes: 4

Milos Lalovic

Reputation: 554

If you mean somthing like:

//INPUT DD DSN=AAAA.BBBB.CCCC.F*,DISP=SHR

to represent:

//INPUT DD DSN=AAAA.BBBB.CCCC.F090622,DISP=SHR
//      DD DSN=AAAA.BBBB.CCCC.F100622,DISP=SHR
//      DD DSN=AAAA.BBBB.CCCC.F110622,DISP=SHR

that can not be done in JCL.