Feedback
Did this article resolve your question/issue?

   

Article

Possible method to fix an overlapped record: Bad record size Records overlap

« Go Back

Information

 
TitlePossible method to fix an overlapped record: Bad record size Records overlap
URL NameP181133
Article Number000132659
EnvironmentProduct: OpenEdge
Version: 10.x, 11.x, 12.x
OS: All supported platforms
Other: DBRPR
Question/Problem Description
Possible method to fix an overlapped record: Bad record size Records overlap
How to fix data corruption due to Invalid data block found. Reason: Bad record size Records overlap
Fixing Records overlap corruption reported by DBRPR Option 1. Report Bad Blocks or PROUTIL -C dbscan 
Steps to Reproduce
Clarifying Information
Error MessageArea extent <> dbkey <dbkey>. Invalid data block found. Reason Bad record size Records overlap
Could not find record. Recid <recid>, error number <errno>, return <ret-code>. (12011)
Defect/Enhancement Number
Cause
Resolution
Whenever there is corruption in a database the first recommendation is always to restore from the last good backup.

What is record block corruption 

The overlapped record in the block reported by DBRPR Option 1. Report Bad Blocks or PROUTIL -C dbscan :
Invalid data block found. Reason: Bad record size Records overlap. 
Can also manifest when the same dbkey is reported by other utilities such with errors relating to invalid record size, invalid table number 
invalid data block found, bad record fragments and overlapped records

for example: DBSCAN may report sequential recids in the corruption messages
a failed binary dump: Could not find record. Recid <recid>, error number <errno>, return <ret-code>. (12011)
a failed probkup: SYSTEM ERROR: Wrong dbkey in block. Found <dbkey>, should be <dbkey2> in area <num>. (1124)
tabanalys reports: Record <recid> in area <area number> has table number 0 which doesn't exist

An overlapped record, is a record that stores the wrong record size in the block header. Further Update or Delete operations on the record cause further corruption to the block because record size information is used in the update/deletion.

This is why DBRPR "Delete record" Options should not be directly used for Bad record size Records overlap corruption:
  • Option 3. Remove Bad Record Fragment
  • Options 1/ 5. Delete Bad Records

Possible method to fix data corruption due to record block corruption 

The following method has been used successfully in a few reported cases to clear the corruption which persist after the system has been rebooted, but ends in a loss of some records.  
  • It is important to reboot in order to ensure this corruption is resident on disk and is not some persistent corruption in the layers between the database shared memory and the disk resident database.
  • To automate DBRPR process the menu options can be fed by an input file: dbrpr.in

    $   proutil <dbname> -C dbrpr < dbrpr.in > dbrpr.out 
    It is recommended to proof the input file against a test database with the same OpenEdge version as DBRPR menu option numbers may vary.

    Additionally, the instructions below include "A" for all database areas. If record block corruption is known and confirmed to be restricted to a particular area only, time can be saved by modifying the input file can be to not select "A" for all areas and instead first drop to OPTION >> 10. Change Current Working Area, and select the related area number presented.  For example, if corruption are only in Area 30:

    >> DBRPR: 1 /
    >> 10, 30
    >> 1,3,7,8,9,G,N

1. Preparation:
  • Run dbanalys or tabanalys to report the current number of records. 
  • Run an OS copy of the current database in the event the following options need to be re-visited or it is determined there are too many corrupted blocks involved to proceed with recovery.
2.  Try to truncate the BI file:
$   proutil <db name> -C truncate bi

If this step was successful then move to Step 4.

3.   When bi recovery is not possible - skip crash recovery

Reconsider reverting to backup or before proceeding, understand what skipping crash recovery means: Open the DBRPR repair menu by forcing access with -F:
$   proutil <db name> -C DBRPR -F 
 
If this step was used, proceed with the Menu Options in Step 4.
 
4.  Scan forwards, (answer NO to "scan backward")
DBRPR: 1 / 1,3,7,8,9,A,G,N
     
$   proutil <db name> -C DBRPR

1. Database Scan Menu
  1. Report Bad Blocks
  3. Fix Bad Blocks
  7. Rebuild Free Chain
  8. Rebuild RM Chain
  9. Rebuild Index Delete Chain
  A. Apply scan to all areas
  G. Go
Scan Backward (Yes/No)? N

5. Repeat Step 4 above, but this time answer: Yes to the scan backward prompt.
DBRPR: 1 / 1,3,7,8,9,A,G,Y 

5a Run DBRPR a 3rd time to only report on Block and Record corruption and pipe to a file
DBRPR: 1 / 1,4,A,G,N
 
$ proutil dbname -C dbrpr

1. Database Scan Menu
  1. Report Bad Blocks
  4. Report Bad Records
    A. Apply scan to all areas
    G. Go
Scan Backward (Yes/No)? N

Note: If a specific Area is scanned as opposed to ALL areas, there is an additional prompt for "Which extent to scan (0=ALL):  where :
0 = all extents in the area or the extent number can be entered for only  a specific extent in the area.report on Block and Record corruption at the prompt . For example if the extent reported is "dbname_30.d67" , only extent 67 can be scanned in area 30 as follows, the area number does not necessarily equate to the DBRPR index number for the area:
DBRPR: 1 / 10, 50,1,4,G,N,67

5b  There may still be "Bad Records" once the Bad record size Records overlap database blocks have been freed up.
These can then be fixed by running DBRPR: 1 / 4,5,A,G and answer NO to scan backwards.

$ proutil dbname -C dbrpr

1. Database Scan Menu
    4. Report Bad Records
    5. Delete Bad Records
    A. Apply scan to all areas
  G. Go
Scan Backward (Yes/No)? N

6. If this worked the last DBRPR scan is clean, take a backup then run a full IDXBUILD.

$   proutil <dbname> -C idxbuild all -B 32000 -TB 64 -TM 32 -TMB 128 -TF 60 -SG 64 -thread 1 -threadnum 4 -mergethreads 4 -datascanthreads 4 -pfactor 90 -rusage

Beginning in 10.2B06 the additional parameters listed above improve the IDXBUILD speed

Finally verify block checksums: 7. Run tabanalys and compare the latest output with the earlier results from the report in the preparation Step 1 to evaluate how many records have been lost as a result.

Since OpenEdge 11.7.0,  DBTOOL Option 3 has been updated to report common errors like invalid record size and invalid table number when all tables are selected and then stop further processing on the corrupt fields. When a specific table is selected,  records that have invalid size or table number are skipped. For further information refer to Article:
Workaround
Notes
Last Modified Date11/20/2020 7:21 AM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.