Feedback
Did this article resolve your question/issue?

   

Article

Can Third Party tools be used to perform online or offline backups ?

« Go Back

Information

 
TitleCan Third Party tools be used to perform online or offline backups ?
URL NameP11011
Article Number000149604
EnvironmentProduct: Progress
Version: 9.x
Product: OpenEdge
Version: 10.x, 11.x, 12.x
OS: All supported platforms
Other: VMWare, VSS, EMC, proquiet, probkup
Question/Problem Description
Can third party tools be used to perform an online backup ?
Are 3rd Party backup solutions supported for OpenEdge Databases?
Can an OS or file system snapshot be used as a reliable database backup
Is OpenEdge compatible with Snapshot Technologies
Does Progress have a service that coordinates the database and snapshot technologies?
Can I use a VMware snapshot as a database backup?
Can I use a EMC snapshot as a database backup?

Can I use a Microsoft Volume Shadow Copy Service snapshot as a database backup?
Why does OpenEdge not support MicroSoft Volume Shadow Copy" based Backups
Is Microsoft Volume Shadow Copy Service (VSS) a supported solution for taking online backups of a PROGRESS database?
If snapshot is taken periodically as a database backup, would crash recovery on a snapshot backup database perform smoothly?
Steps to Reproduce
Clarifying Information
Error Message
Defect Number
Enhancement Number
Cause
Resolution
Yes, filesystem snapshot technologies, OS or similar non-Progress backup utilities can be used to take an online or offline backup of a Progress OpenEdge database, provided these do not use any block level or incremental methodologies. 

While Progress does not provide an integrated service or deamon for snapshot technologies, the PROQUIET utility can specifically be used to facilitate the use of these against an online database instance. The only online backup method supported is our PROBKUP online or the use of the PROQUIET utility for OS copies (which 3rd party backup products fall under).  The 3rd Party backup utilities themselves are not supported by Progress, they are supported by their Vendor.

What happens when a 3rd Party utility backs up an online Progress database?
 
When a snapshot is taken while a database is running, any of the following situation may be occurring:
  1. Active Transactions are running;
  2. Shared memory has not been flushed down to disk;
  3. Internal time stamps mismatch between database files and BI notes. 
When the snapshot is taken when a database is running, all the database activity or changes may not be included in this snapshot.  This is because changes are occurring in shared memory and a snapshot would not take that into account.  A snapshot of a database results in a database that is in a crashed status.  The database has to go through crash recovery to backout all incomplete transactions that were running at the time of the snapshot and replay others that were recorded in the last 4 bi clusters that were not yet flushed to disk.
 
There is a definite possibility of errors that are not totally fatal but might leave the database in an inconsistent state. There is no 100% guarantee that the database can be recovered. The recommendation to address the concerns above is to quiet the database prior to taking a snapshot.  See Options listed below.

Typically, problems reported are database down situations with 9446 1124 error messages as a result, essentially caused by a 3rd Party product accessing the run-time Progress environment to take the backup.  The existing database usually succeeds crash recovery and data integrity has not been compromised as a result of this concurrency unless skipping crash recovery is needed, at which time the 3rd Party backup is restored.
 
SYSTEM ERROR: <function>: Bad file descriptor was used during <system call>, fd  <file descriptor>, len <bytes>, offset <bytes>, file <file-name>. (9446)
SYSTEM ERROR: wrong dbkey in block. Found <dbkey>, should be <dbkey2> (1124)

Will online 3rd Party backup be reliable?

If a 3rd Party 'online backup' against an OpenEdge database that was not first quiesced is ever be needed, it is then a 'crashed' database that is being recovered and this cannot always be guaranteed.
  • Internal time stamps mismatch between database files and BI notes may prevent access to the database without forcing in
  • Any open transactions will need to be rolled back as part of the BI recovery process
  • There's always the chance that if the database blocksize is larger than that of the filesystem, torn pages could result.
  • The data physically on disk is not anything like the data in a running database's shared memory (assuming production and reasonable workload). 
 
Will an offline 3rd Party backup be reliable? 

Yes, an offline 3rd Party backup will be reliable as long as there is nothing else touching the database at the time the backup was taken.

Typically cron jobs or scheduled tasks are left running when the database is shut down. If the database is ever used in single user mode, then do not attempt to use a non-Progress backup to copy the files on disk as they are dynamically being updated and the non-Progress tool will most likely catch the files midstream causing the resultant files to more than likely be out of sync.

Typically the filesystem directory name is temporarily changed to ensure that the files are not findable to be accessed while offline 3rd Party backups are being created.
 
How to use 3rd Party utilities to backup an online Progress database?

Option 1:   Ensure that the 3rd Party utility does not access the run-time environment.
 
Use PROBKUP to create backup volumes of OpenEdge databases online. Only while the Before-Image files are being backed up is update activity restricted against the database. As soon as Data Blocks are updated, online transaction processing resumes. The PROBKUP utility is fully integrated with After-Imaging and OpenEdge Replication environments. Additionally incremental, multiple volume and redundancy to add to the backup for error correction can be used.

Once the PROBKUP has completed, move the resulting backup volume(s) elsewhere, where they can then copied by the 3rd party utility, excluding all online database files from the utility.

Option 2:   Use the PROQUIET utility.
 
The PROQUIET utility is available with the Enterprise Database License and can be used to take reliable online 3rd Party database backups.

When a PROQUIET is enabled against a database, (PROQUIET enable) all activity on the database will be halted and buffers in shared memory will be flushed to disk.  Prior to taking a snapshot of a quieted database, validate via the database log file that the database has reached a "quiet" state before proceeding with the snapshot. It is imperative the the quiet point is raised before the copy utility is allowed to continue so that the database is not flushing data to the disks.

Typically 3rd Party backup utilities are preferred due to the fact that the snapshot technology is fast, however during the database quiesce time all activity except read access to the the database will be frozen, therefore try to avoid this operation while the database is being highly used or while there are many users connected.

Once the database buffers have been synchronised to disk, the 3rd Party backup takes the database copy along with whatever else it takes during database quiesce time.

Once the 3rd Party backup is complete the quiet point can be released (PROQUIET disable).

In any event, as with any DR (disaster recovery) plan, 3rd Party solutions should not be implemented until fully tested.  This means not only testing the method to backup or take a snapshot of a running database, but also to recover or restore the database from the backup or snapshot taken and ensure a documented DR plan is available when required.  Our Progress Professional Services team can further assist if required. Contact your Progress Account Manager to discuss these options.
Workaround
Notes
Last Modified Date9/22/2021 8:26 PM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.