Feedback
Did this article resolve your question/issue?

   

Article

Why the .bi file grows larger than original bi with roll forward?

« Go Back

Information

 
TitleWhy the .bi file grows larger than original bi with roll forward?
URL NameP3622
Article Number000145295
EnvironmentProduct: Progress OpenEdge
Version: All supported versions
OS: All supported platforms
Question/Problem Description
Why the .bi file grows larger than original bi with roll forward?
Why applying after image files causes the BI file to grow?
Is the Before Image (BI) file used on a standby database?
Why is the before image area on the roll forward target database growing beyond its fixed extents?
Why is the bi file on the target database growing as transaction notes are applied?
Steps to Reproduce
Clarifying Information
Error Message
Defect Number
Enhancement Number
Cause
Resolution

In order to explain how transactions are written to a .bi file and to the .ai file, and what may happen to a .bi file when roll forwarding with an .ai file, the following symbols and diagrams will be used to represent notes, clusters.

There are 4 notes written to a .bi and .ai file for each transaction. These are:

  1. Begin note
  2. Pre note (snapshot of record before change; can be big)
  3. Post note (changes to record)
  4. End note

Each note will be represented by N<n>, where "n" represents either note 1 (Begin), note 2 (pre), note 3 (post) or note 4 (end).

A transaction will be represented with a T<n>, where "n" is the transaction number.

Boxes will represent BI Clusters:

-------------------------------------------------------------------------------
|- Cluster 1        |- Cluster 2        |- Cluster 3        |- Cluster 4        |
-------------------------------------------------------------------------------


Typically a pre note follows closely after a begin note, and an endnote follows closely after a post note. Therefore, in a multi-user multi-threaded environment, a bi cluster could contain the following information:

T1N1 T1N2 T2N1 T1N3 T2N2 T1N4

This would represent:
Transaction 1 with a begin and pre note, Transaction 2 with a begin note, followed by Transaction 1's post note, Transaction 2's pre note, then Transaction 1's end note.

Assuming these transaction notes were all contained in one bicluster, this would mean Transaction 1 was complete, but Transaction 2 is still under way. This bicluster is not available for reuse until Transaction 2 is finished.

If the .bi file had just been truncated and a fresh .ai extent, the first four biclusters would contain the same information as the ai notes until the fourth bicluster was full (given that are the same size). This is the point where the contents of the .ai and the .bi file will begin to look different as biclusters become available for re-use. In the previous example, once Transaction 2 was finished, the first cluster of the .bi file would be available for re-use.

For a picture of this, assume the following 5 multi-threaded sessions are taking place:

T1  |.......N3.N4
T20 |..N1.N2.N3.N4
T21 |...................N1........N2...................N3.N4
T22 |.....................N1.N2....N3.N4
T8  |N3.N4
------------------------------------------------->
time



The .bi file could look something like this:

---------------------------------------------------------------------------------------------------------
T1N1 T1N2...|   ...        |...T8N3 T20N1 T8N4 T20N2 T1N3 T20N3 T1N4 T20N4  |  T21N1 T22N1 T22N2 T21N2  |
---------------------------------------------------------------------------------------------------------
 cluster1                                             Cluster 3                   Cluster 4


Cluster 3 has the end note for Transactions 8 and 1, plus the begin and end notes for Transaction 20. There are no other "open" transactions residing in this cluster or in the earlier cluster 1 and cluster 2. Therefore, these three clusters are available for reuse. At this point, we are ready to write T22N3 to the .bi file. Rather than extending the .bi file with another cluster, Progress will reuse cluster 1. The clusters will look like this after we put the remaining notes into the .bi file:

-------------------------------------------------------------------------------------------------------------------
T22N3 T22N4 T21N3 T21N4  |    ...       |...T8N3 T20N1 T8N4 T20N2 T1N3 T20N3 T1N4 T20N4 | T21N1 T22N1 T22N2 T21N2 |
-------------------------------------------------------------------------------------------------------------------

 Cluster1                                             Cluster 3                                 Cluster 4


However, the .ai file is a sequential file, and will not reuse clusters. "Cluster" is used in the diagram simply to assist in aligning the example. The corresponding sequential .ai file's transaction notes look like this:

-----------------------------------------------------------------------------------------------------------------------------------------------
T1N1 T1N2...|   ...        |...T8N3 T20N1 T8N4 T20N2 T1N3 T20N3 T1N4 T20N4  |  T21N1 T22N1 T22N2 T21N2  | T22N3 T22N4 T21N3 T21N4 
-----------------------------------------------------------------------------------------------------------------------------------------------
 Cluster1                                             Cluster 3                   Cluster 4                    Cluster 5

At this point, the .bi and the .ai files are no longer the same length. The .ai file continues to grow sequentially while the .bi file is constantly taking advantage of the reusable clusters in the .bi file.

It is important to note that the .bi file is not allowed to reuse a cluster until it has aged at least 60 seconds by default, and can be overridden with the -G parameter. Since application users will stop to answer a phone, get a cup of coffee, shuffle papers, etc., writing to the .bi file by interactive users will not necessarily be a constant stream. This aids in the aging of clusters in the .bi file.

These two factors: aging of clusters and the rate at which the .bi file is being written to during normal processing is important to remember for the journey into roll-forwarding.


ROLL FORWARDING: THE AFFECT ON THE .BI FILE

Let's assume we now have stopped using this .ai file and are ready to apply it to a previous backup. The .bi file during this day's activity yielded a .bi file of approximately 8MB.
Will the .bi file that's created during roll forward create a .bi file of the same size, smaller, or larger? It would not be unlikely at all to have the .bi file created during roll forward to be larger, and this is why:

Using the .ai file above, the .bi file notes the transactions as they are applied to the database with RFUTIL -C roll forward. They are applied in rapid succession, as we are not talking hand-entered data from our previous real-time experience. By the time cluster 4 is full, cluster 1 cannot be reused, as proper aging has not occurred for Cluster 1. Therefore, the .bi file will be extended, and the information will be written to cluster 5. 

Additionally, having different cluster sizes for your .bi and .ai files could also further impact how big the .bi will grow during roll forward.

SUMMARY:

An infrequent occurrence of .bi file growth to many times the size of the .ai file during roll forward recovery is not usually an indicator of required tuning with respect to modification of bi blocks, bi cluster, or database block sizes. Each roll forward of the new ai file will cause the bi file to grow in very different patterns as opposed to the previous ai roll forward. This is due to the very differently recorded sequence of events.
Workaround
Notes
Last Modified Date9/13/2015 11:46 AM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.