Maintaining data integrity and consistency is paramount in real-time data replication. However, users of Oracle GoldenGate may encounter the OGG-02191 error, signaling an incompatible record issue. In this post, we’ll delve into the intricacies of this error code, explore its potential causes, and provide strategies to troubleshoot and resolve it effectively, ensuring seamless data replication operations.
The OGG-02191 error signifies that the GoldenGate Extract process encountered a record that it could not process due to incompatibility. This error can occur when the structure or format of the record differs from what GoldenGate expects, disrupting the replication process and halting Extract operations.
Incompatibilities leading to the OGG-02191 error can stem from various sources within the data replication environment. Changes to the source database schema, such as alterations in column structures or the addition/removal of fields, may result in records that deviate from GoldenGate’s expected format. Data type disparities between source and target databases can also contribute to compatibility issues, particularly when the data type of a column in the source differs from its counterpart in the target. Moreover, encounters with unsupported data formats or structures can trigger errors during replication, disrupting the process. Variations in character encoding between systems further compound these challenges, potentially causing discrepancies in record interpretation. Additionally, records containing intricate data structures, such as nested arrays or complex JSON objects, may pose compatibility hurdles for GoldenGate’s replication logic.
This is the trial record of corruption issues. There are two options to handle it.
OPTION 1: Skip the bad record and start the replicate
OPTION 2: Automated Trail File Recovery stops the pump process and removes the corrupted remote trail and the consecutive remote trail files from the corrupted remote trail by leaving the last healthy remote trail file. It’s a Golden Gate 12.2 new feature.
Option 1: Skip the bad record:
REPLICAT abends with error ERROR OGG-02171 Error reading LCR from the data source. Need to skip the bad record in a file, and restart REPLICAT
2021-04-28 15:47:29 INFO OGG-02243 Opened trail file /MYSQL/ggtrail/PRPAY1/mp000000002 at 2021-04-28 15:47:29.963515.
2021-04-28 15:47:50 ERROR OGG-02171 Error reading LCR from data source. Status 509, data source type TrailDataSource.
2021-04-28 15:47:50 ERROR OGG-02191 Incompatible record 101 in /MYSQL/ggtrail/PRPAY1/mp000000002, rba 696,172 when getting trail header.
2021-04-28 15:47:50 INFO OGG-02333 Reading /MYSQL/ggtrail/PRPAY1/mp000000002, current RBA 696,172, 0 records, m_file_seqno = 0, m_file_rba = 696,172.
Solution:
Use the logdump utility to get the next record position. You need to skip the bad record in the file and restart REPLICAT. First, position the position on the failed record. The logdump tool will display the next record. Optionally, get the next+1 record. Then, set REPLICAT to start at the next record.
ALTER RRPAY1, EXTSEQNO 2, EXTRBA
[oracle@ORA-X1 kpay1]$ cd $OGG_HOME
[oracle@ORA-X1 gguser]$ pwd
/MYSQL/gguser
[oracle@ORA-X1 gguser]$ ./logdump
Oracle GoldenGate Log File Dump Utility for Oracle
Version 19.1.0.0.4 OGGCORE_19.1.0.0.0_PLATFORMS_191017.1054
Copyright (C) 1995, 2019, Oracle and/or its affiliates. All rights reserved.
Logdump 2 >open /MYSQL/ggtrail/kpay1/mp000000002
Logdump 3 >ghdr on
Logdump 4 >detail on
Logdump 5 >pos 696172
Reading forward from RBA 696172
Logdump 6 >scanforheader
___________________________________________________________________
Hdr-Ind : E (x45) Partition : L (x4c)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 962 (x03c2) IO Time : 2020/10/26
0:19:07.005.706
IOType : 135 (x87) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 200132 AuditPos : 122219168
Continued : N (x00) RecCount : 1 (x01)
2021/04/28 10:19:07.005.706 GGSUnifiedPKUpdate Len 962 RBA 696345
Name: OGG.GG_HEARTBEAT_SEED (TDR Index: 2)
After Image: Partition x4c G s
df01 0000 0000 0700 0000 0300 4947 5401 001f 0000 | ............IGT.....
0032 3032 302d 3130 2d32 363a 3130 3a31 383a 3037 | .2021-04-28:10:18:07
2e32 3530 3233 3330 3030 0200 0400 ffff 0000 0300 | .250233000..........
0400 ffff 0000 0400 0400 ffff 0000 0500 0400 ffff | ....................
0000 0600 1f00 ffff 0000 0000 0000 0000 0000 0000 | ....................
0000 0000 0000 0000 0000 0000 0000 0000 0007 001f | ....................
00ff ff00 0000 0000 0000 0000 0000 0000 0000 0000 | ....................
Before Image Len 483 (x000001e3)
BeforeColumnLen 479 (x000001df)
Column 0 (x0000), Len 7 (x0007)
Column 1 (x0001), Len 31 (x001f)
Column 2 (x0002), Len 4 (x0004)
Column 3 (x0003), Len 4 (x0004)
Column 4 (x0004), Len 4 (x0004)
Column 5 (x0005), Len 4 (x0004)
Column 6 (x0006), Len 31 (x001f)
Column 7 (x0007), Len 31 (x001f)
Column 8 (x0008), Len 31 (x001f)
Column 9 (x0009), Len 31 (x001f)
Column 10 (x000a), Len 12 (x000c)
Column 11 (x000b), Len 14 (x000e)
Column 12 (x000c), Len 4 (x0004)
Column 13 (x000d), Len 31 (x001f)
Column 14 (x000e), Len 31 (x001f)
Column 15 (x000f), Len 31 (x001f)
Column 16 (x0010), Len 31 (x001f)
Column 17 (x0011), Len 4 (x0004)
Column 18 (x0012), Len 14 (x000e)
Column 19 (x0013), Len 14 (x000e)
Column 20 (x0014), Len 31 (x001f)
After Image Len 479 (x000001df)
Column 0 (x0000), Len 7 (x0007)
Column 1 (x0001), Len 31 (x001f)
Column 2 (x0002), Len 4 (x0004)
Column 3 (x0003), Len 4 (x0004)
Column 4 (x0004), Len 4 (x0004)
Column 5 (x0005), Len 4 (x0004)
Column 6 (x0006), Len 31 (x001f)
Column 7 (x0007), Len 31 (x001f)
Column 8 (x0008), Len 31 (x001f)
Column 9 (x0009), Len 31 (x001f)
Column 10 (x000a), Len 12 (x000c)
Column 11 (x000b), Len 14 (x000e)
Column 12 (x000c), Len 4 (x0004)
Column 13 (x000d), Len 31 (x001f)
Column 14 (x000e), Len 31 (x001f)
Column 15 (x000f), Len 31 (x001f)
Column 16 (x0010), Len 31 (x001f)
Column 17 (x0011), Len 4 (x0004)
Column 18 (x0012), Len 14 (x000e)
Column 19 (x0013), Len 14 (x000e)
Column 20 (x0014), Len 31 (x001f)
Logdump 7 >n
___________________________________________________________________
Hdr-Ind : E (x45) Partition : . (x0c)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 271 (x010f) IO Time : 2020/10/26
0:19:15.005.532
IOType : 5 (x05) OrigNode : 255 (xff)
TransInd : . (x00) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 200132 AuditPos : 131688464
Continued : N (x00) RecCount : 1 (x01)
2021/04/28 10:19:15.005.532 Insert Len 271 RBA 697424
Name: RKPAYADMIN.BARCODE_FOR_SHOPPER (TDR Index: 1)
After Image: Partition
0c G b
0000 1300 0000 0f00 3130 3330 3130 3438 3130 3030 |
.......103010481000
3030 3001 0009 0000 0005 0033 3635 3839 0200 0600 |
00........36589....
0000 0200 3136 0300 0500 0000 0100 3004 0005 0000 | ....16........0.....
0001 0030 0500 0500 0000 0100 3006 0005 0000 0001 | ...0........0.......
0030 0700 0500 0000 0100 3008 000a 0000 0000 0000 | .0........0.........
0000 0000 0009 000a 0000 0000 0000 0000 0000 000a | ....................
0015 0000 0032 3032 302d 3130 2d32 363a 3130 3a31 | .....2021-04-28:10:1
Column 0 (x0000), Len 19 (x0013)
Column 1 (x0001), Len 9 (x0009)
Column 2 (x0002), Len 6 (x0006)
Column 3 (x0003), Len 5 (x0005)
Column 4 (x0004), Len 5 (x0005)
Column 5 (x0005), Len 5 (x0005)
Column 6 (x0006), Len 5 (x0005)
Column 7 (x0007), Len 5 (x0005)
Column 8 (x0008), Len 10 (x000a)
Column 9 (x0009), Len 10 (x000a)
Column 10 (x000a), Len 21 (x0015)
Column 11 (x000b), Len 10 (x000a)
Column 12 (x000c), Len 7 (x0007)
Column 13 (x000d), Len 10 (x000a)
Column 14 (x000e), Len 21 (x0015)
Column 15 (x000f), Len 4 (x0004)
Column 16 (x0010), Len 4 (x0004)
Column 17 (x0011), Len 4 (x0004)
Column 18 (x0012), Len 4 (x0004)
Column 19 (x0013), Len 4 (x0004)
Column 20 (x0014), Len 10 (x000a)
Column 21 (x0015), Len 5 (x0005)
Logdump 8 >
Logdump 8 >exit
./ggsci
GGSCI (qanfv-1-dbs-1b) 2> AALTER Rkpay1, EXTSEQNO 2, EXTRBA 69634
2021-04-28 15:58:14 INFO OGG-06594 Replicat Rkpay1 has been altered. Even the start up position might be updated, duplicate suppression remains active in next startup. To override duplicate suppression, start Rkpay1 with NOFILTERDUPTRANSACTIONS option.
REPLICAT altered.
GGSCI (ORA-X1) 28> START Rkpay1
Sending START request to MANAGER ...
REPLICAT Rkpay1 starting
GGSCI (ORA-X1) 28> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
JAGENT RUNNING
PMSRVR RUNNING
EXTRACT RUNNING EORACLE 00:00:07 00:00:02
EXTRACT RUNNING ESORACLE 00:00:01 00:00:01
EXTRACT RUNNING PORACLE 00:00:00 00:00:01
EXTRACT RUNNING PSORACLE 00:00:00 00:00:02
REPLICAT RUNNING RRMYSQL 00:00:00 00:00:01
REPLICAT RUNNING Rkpay1 00:00:00 00:00:04
Option 2: Automated Trail file recovery new feature from OGG 12.2
Solution:
First, Stop the pump from the source side; delete all the trail files from the corrupted sequence
Start the pump in the source, and any missing trail files will be automatically rebuilt after the bouncing of the pump.
Start the replicate process
