Feedback
Did this article resolve your question/issue?

   

Article

Consequences of using the -F startup option

« Go Back

Information

 
TitleConsequences of using the -F startup option
URL NameP24330
Article Number000128637
EnvironmentProduct: OpenEdge
Version: All supported versions
OS: All supported platforms
Question/Problem Description
Consequences of using the -F startup option
The Perils of using the Force (-F) option to start the database.
How to force access into a damaged database?
How to skip crash recovery on a damaged database?
Steps to Reproduce
Clarifying Information
Error Message
Defect/Enhancement Number
Cause
Resolution
Overview

The -F option is provided as a last resort emergency measure and should never be needed otherwise. This parameter is made available so that as much data as possible can be salvaged from a database that is damaged and otherwise unusable. It should be used only when the alternative is loss of the entire database.  It cannot be stressed strongly enough that using -F must be considered as the very last resource. Reverting to a backup is the most reliable and time efficient option.

The discussion below does not apply to when the -F option is used with PROSHUT or PROMON. The effect of forcing the database to shutdown is outlined in Article  "What happens during PROSHUT -by and PROSHUT -F?"   

Some background information

Always think of the Progress database as the sum of three elements:
  • All the data files: Control Area: .db; Storage Area extents: .d1, .d2, etc.; 
  • Before-Image file(s): .b1, .b2, etc.) and
  • The data resident in shared memory.
When a database is abnormally shut down (for example, in case of a power failure), the data resident in memory is lost along with those updates that had not yet been written to disk.  When the database is started up again, the rollback process takes place; the rollback consists of two parts:
 
1. The BI Redo Phase:
 
Some transactions that had been committed at the time the database crashed might have been flushed to the before-image file, but not to the associated data files because of how Progress optimizes writes to the data files (.d1, d2, etc.). This is completely normal. During the redo phase Progress scans the two most-recently used before-image clusters to make sure that all committed transactions are indeed flushed to the data files. The start and the end of the redo phase are marked by informational messages 5326 and 7161 respectively; these message appear when the database is next accessed:
 
Begin Physical Redo Phase at  <address>. (5326)
Physical Redo Phase Completed at blk <blk> off <off> upd <upd>. (7161)


2. The BI Physical and Logical Undo Phases (Rollback):

Once the redo phase is complete, Progress scans the before-image file backwards searching for transactions that were started but not committed at the time the database crashed. Any incomplete transaction is rolled back and this is possible thanks to the information in the before-image file itself.

Forcing access with -F

When access is forced into a database with the -F option, both the redo phase and the rollback are skipped; also, the before-image is rebuilt from scratch. The information needed by the redo phase and the rollback is permanently lost. Effectively rollback recovery is skipped by forcing into the database.

As a consequence, the database is left in an inconsistent state:
  • Some of the transactions that were committed at the time of the crash might not be flushed to the data files or, even worse, they might be left partially committed because the redo phase was skipped. This is the reason why a database might be damaged by using -F even if there were no transactions active at the time the database crashed because the data physically on disk is rarely like the data in a running database.
  • Transactions which were uncommitted at the time of the crash will not be undone back because rollback recovery was skipped, which leaves logical application corruption.
Consequences of skipping crash recovery using the Force option (-F) include:
  • Records updated incompletely.
  • Records deleted partially.
  • Record fragments which refer to non-existent fragments.
  • Records without index entries.
  • Index entries that refer to non-existent records.
  • Inconsistent Free and RM chains (used internally to allocate space for new records).
  • Logical inconsistencies due to the partially completed transactions.
  • After imaging is disabled on databases that have it enabled, when the FORCE (-F) parameter is used.
The fact that a database is damaged is usually not immediately apparent. For instance, the existence of a partially created record is revealed only when that record is accessed and an error occurs. Until then, everything may appear normal.

Physical inconsistencies in index structures can be repaired by rebuilding the indexes with the offline index rebuild utility. This is possible because the index rebuild utility first deletes all index blocks and then creates new ones using the key values present in existing records assuming no record corruption has resulted as a consequence.

Critical schema indexes however, will not be repaired in this way; the implication is that the database schema is corrupted, and information cannot be reliably pulled out from the database even by dumping the database. This is a worst-case scenario, but it's possible.

On the other hand, physical inconsistencies in data records (including the records that describe the schema) cannot be repaired. The only possibility is to remove damaged records manually (if they can be located), resulting in data loss.

Logical inconsistencies can be repaired only by an examination of all data values in all records and a verification that they are correct, that parent-child relationships are valid, that no records are missing and that there are no duplicate records. This is usually a daunting and time consuming task.

For a detailed example of the above, Refer to Article  Consequences of skipping crash recovery using the Force option (-F)    

Under these conditions, reverting to a backup is probably a better option than trying to repair a database damaged by forcing access with -F.

Using -F

First of all, backup your database now. Try to use PROBKUP first, but if PROBKUP crashes because a database block is damaged too severely, resort to backing up the database with OS utilities.

After the backup, force access to the database by using the -F option with the PROUTIL utility in conjunction with the truncate bi qualifier:
$   proutil <dbname> -C truncate bi -F

The following messages and prompt appear:
The -F option has been specified to proutil. (6260)
Forcing into the database skips database recovery. (6261)
This leaves the database in unknown state, considered damaged. (6262)
Are you sure you want to skip crash recovery? (Y/N) (6263)

Entering "y" for "yes", forced access occurs and the following additional dire warnings are displayed and written to the log file:
** The FORCE option was given, database recovery will be skipped. (33)
After-imaging disabled by the FORCE parameter. (4790)
** Your database was damaged. Dump its data and reload it. (37)

Now the database must be dumped and reloaded into a new database structure.

Prior to embarking upon this activity an IDXBUILD of all primary indexes in an attempt to repair the damage on those indexes in order that as much data as possible can be dumped. Please bear in mind that in this case the backup is even more important, because if anything goes wrong during IDXBUILD, the only way to recover the database is to restore it from backup.

  How to list the primary index for all tables?  

Finally: Dump the database with the Data Dictionaries utilities and then load it via the bulkload utility.

Binary dump is discouraged under these circumstances, because while binary dump is quicker, it checks record integrity to a very limited extent; therefore a corrupted record may well be dumped "as is" and then be loaded into the new database without any fix; as a consequence the new database would be corrupted as well.

The physical integrity of the database will be restored at this point for the data loaded. Logical integrity however will still need to be ascertained from an application perspective.
 
Workaround
Notes
Last Modified Date11/20/2020 7:31 AM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.