Remove duplicates on specific record type using JCL sort

Question

can you please help me to achieve below goal JCL sort ? I have input file with different set if records. I need to remove the duplicates only specific record type based on 10,04 position. There should not be any change in other types of records.

I need to remove duplicates oly for P4 on position 10,4.

iNPUT:

P1AAAAAAA0101xxxxxxx

P2AAAAAAA0101xxxxxxx

P4AAAAAAA0101xxxxxxx

P4AAAAAAA0101xxxxxxx

P4AAAAAAA0102xxxxxxx

P4AAAAAAA0103xxxxxxx

P4AAAAAAA0104xxxxxxx

P5AAAAAAA0101xxxxxxx

P7AAAAAAA0101xxxxxxx

P9AAAAAAA0101xxxxxxx

EXPECTED OUTPUT:

P1AAAAAAA0101xxxxxxx

P2AAAAAAA0101xxxxxxx

P4AAAAAAA0101xxxxxxx

P4AAAAAAA0102xxxxxxx

P4AAAAAAA0103xxxxxxx

P4AAAAAAA0104xxxxxxx

P5AAAAAAA0101xxxxxxx

P7AAAAAAA0101xxxxxxx

P9AAAAAAA0101xxxxxxx

Answer 1

Here you go.

//JOBNAME  JOB 1,NOTIFY=&SYSUID            
//STEP1    EXEC PGM=SORT                                 
//SORTIN   DD *                                          
P1AAAAAAA0101XXXXXXX                                     
P2AAAAAAA0101XXXXXXX                                     
P4AAAAAAA0101XXXXXXX                                     
P4AAAAAAA0101XXXXXXX                                     
P4AAAAAAA0102XXXXXXX                                     
P4AAAAAAA0103XXXXXXX                                     
P4AAAAAAA0104XXXXXXX                                     
P5AAAAAAA0101XXXXXXX                                     
P7AAAAAAA0101XXXXXXX                                     
P9AAAAAAA0101XXXXXXX                                     
//SORTOUT  DD SYSOUT=*                                   
//SYSOUT   DD SYSOUT=*                                   
//SYSIN    DD *                                          
 INREC IFTHEN=(WHEN=INIT,BUILD=(1:1,20,21:SEQNUM,2,ZD)), 
       IFTHEN=(WHEN=(1,2,CH,EQ,C'P4'),OVERLAY=(21:C'01'))
 SORT FIELDS=(1,2,CH,A,10,4,CH,A,21,2,CH,A)              
 SUM FIELDS=NONE                                         
 OUTREC FIELDS=(1:1,20) 

• I've used INREC IFTHEN to set sequence numbers at the end of the record. Sequence numbers are set to 1 for records with first 2 bytes as P4.
• Control fields included in the SORT FIELDS statement are first 2 bytes, col 10 thru 4 bytes and col 21 thru 2 bytes (sequence number).
• Note that sequence numbers for duplicate records in P4 will be 1. So SUM FIELDS=NONE would remove just those records.

Output:

P1AAAAAAA0101XXXXXXX
P1AAAAAAA0101XXXXXXX
P2AAAAAAA0101XXXXXXX
P4AAAAAAA0101XXXXXXX
P4AAAAAAA0102XXXXXXX
P4AAAAAAA0103XXXXXXX
P4AAAAAAA0104XXXXXXX
P5AAAAAAA0101XXXXXXX
P7AAAAAAA0101XXXXXXX
P9AAAAAAA0101XXXXXXX 

Let's run the same set of SORT statements with following input. Note that first 2 records (P1) are duplicates along with P4.

//JOBNAME  JOB 1,NOTIFY=&SYSUID                           
//STEP1    EXEC PGM=SORT                                  
//SORTIN   DD *                                           
P1AAAAAAA0101XXXXXXX                                      
P1AAAAAAA0101XXXXXXX                                      
P2AAAAAAA0101XXXXXXX                                      
P4AAAAAAA0101XXXXXXX                                      
P4AAAAAAA0101XXXXXXX                                      
P4AAAAAAA0102XXXXXXX                                      
P4AAAAAAA0103XXXXXXX                                      
P4AAAAAAA0104XXXXXXX                                      
P5AAAAAAA0101XXXXXXX                                      
P7AAAAAAA0101XXXXXXX                                      
P9AAAAAAA0101XXXXXXX                                      
//SORTOUT  DD SYSOUT=*                                    
//SYSOUT   DD SYSOUT=*                                    
//SYSIN    DD *                                           
 INREC IFTHEN=(WHEN=INIT,BUILD=(1:1,20,21:SEQNUM,2,ZD)),  
       IFTHEN=(WHEN=(1,2,CH,EQ,C'P4'),OVERLAY=(21:C'01')) 
 SORT FIELDS=(1,2,CH,A,10,4,CH,A,21,2,CH,A)               
 SUM FIELDS=NONE                                          
 OUTREC FIELDS=(1:1,20)                                   

Output after running the job is shown below. Note that duplicate P1 record are preserved and duplicate records with specific record type (P4) is only removed.

P1AAAAAAA0101XXXXXXX
P1AAAAAAA0101XXXXXXX
P2AAAAAAA0101XXXXXXX
P4AAAAAAA0101XXXXXXX
P4AAAAAAA0102XXXXXXX
P4AAAAAAA0103XXXXXXX
P4AAAAAAA0104XXXXXXX
P5AAAAAAA0101XXXXXXX
P7AAAAAAA0101XXXXXXX
P9AAAAAAA0101XXXXXXX

Hope this helps.

Answer 2

I would like to present you a small solution, a Little complicated but it works, maybe someone will have another one, more compact than mine:

//S1       EXEC  PGM=ICETOOL                                    
//TOOLMSG  DD SYSOUT=*                                          
//DFSMSG   DD SYSOUT=*                                          
//IN       DD *                                                 
P1AAAAAAA0101xxxxxxx                                            
P2AAAAAAA0101xxxxxxx                                            
P4AAAAAAA0101xxxxxxx                                            
P4AAAAAAA0101xxxxxxx                                            
P4AAAAAAA0102xxxxxxx                                            
P4AAAAAAA0103xxxxxxx                                            
P4AAAAAAA0104xxxxxxx                                            
P5AAAAAAA0101xxxxxxx                                            
P7AAAAAAA0101xxxxxxx                                            
P7AAAAAAA0101xxxxxxx                                            
P9AAAAAAA0101xxxxxxx                                            
//T1       DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T2       DD DSN=&&T2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//T3       DD DSN=&&T3,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT      DD SYSOUT=*                                          
//TOOLIN   DD *                                                 
 COPY     FROM(IN)    TO(T1)  USING(AST1)                       
 COPY     FROM(IN)    TO(T2)  USING(AST2)                       
 SELECT   FROM(T1)    TO(T3)  ON(10,4,CH) NODUPS                
 MERGE    FROM(T2,T3) TO(OUT) USING(AST3)                       
//AST1CNTL DD *                
 INCLUDE COND=(1,2,CH,EQ,C'P4')
//AST2CNTL DD *                
 INCLUDE COND=(1,2,CH,NE,C'P4')
//AST3CNTL DD *                
 MERGE FIELDS=(1,14,CH,A)

Regards, Andreas