BICLUSTERS Performance Improvement
BIGROW uses a buffered file handle
to grow the bi file. The buffered file handle does unbuffered I/O
which extends the bi file with formatted bi clusters. It is this which
which adds additional time to the sync bi file on completion.OpenEdge 11.3.0
reworked this area on Solaris
- BI extents written are flushed from the file system cache with a fdatasync() call and buffered I/O is only used again once the bi clusters grow/format has finished.
- This rework extends to the runtime database when for example, a long running transaction necessitates new bi-clusters need to be added to the double-linked BI Chain at Checkpoint time.
This performance improvement varied on some platforms but not as much on others, but overall better.
In OpenEdge 11.3
- Windows and Linux still extend files unbuffered.
- To improve the performance it's advisable to pre-grow the BI file before starting the database to site-specific bi sizes, thereby negating the need to extend the bi file at runtime.
- Adding the non-raw (-r) parameter will improve the BIGROW time by not needing to sync the bi file writes from OS cache when the grow/format is complete: proutil <dbname> -C bigrow <n> -r
- By not needing to sync the bi files when the grow/format is complete, this gives an indication of approximately the difference that could be achieved with/without the additional time it takes to sync the bi files on completion (without the non-raw parameter).
In OpenEdge 11.7.6
and OpenEdge 12.1
, all certified platforms
This bi cluster performance optimization was reworked and made available on all platforms
. All supported platforms were enhanced to improve performance times when extending the bi file with formatted clusters. Article:
The "Furgal Test"
Another use-case for BIGROW is what has commonly been referred to as the "Furgal Test"
, to measure disk speed specifically by timing how long it takes to grow a bi file of a certain size on disk. This benchmark provides a quick and easy means to uniformly gather information about synchronous IO performance. The results enable:
a) Comparative BIGROW performance results with other environments when investigating bottlenecks on I/O or CPU that are impacting our ability to write changes to disk in a timely fashion. For example when Checkpoint metrics show Sync Time as a fair proportion of the Checkpoint Duration timings.
b) A litmus test of sorts. While bigrow is sequential and database I/O is completely random, experience has shown a strong correlation between long times to 'simply' preformat and extend the bi cluster chain and poor application/database performance.
c) To keep an eye on infrastructure trends that influence OpenEdge performance. This (historic or environment comparative ) evidence provides a reliable indicator of storage implementation and/or configuration issues which lead back to database/application performance problems. When storage vendors deny changing anything, this claim can confidently be rebutted with at least the "Furgal test" metrics.
In order to execute this unbuffered I/O
test, the improvement described above needs to be disabled by using an undocumented parameter: -zextendSyncIO.
This parameter is not ever expected to be needed otherwise, unless bigrow
is being used as a benchmark on platforms running an OpenEdge version with this performance improvement.Example: "Furgal Test" re-format 96 MB disk space to pre-allocate 6 bi clusters
- To verify a minimum of 10 MB/Second sequential write speed measured using a BIGROW benchmark
- Add the 4 default bi clusters to the bi grow value specified, to pre-grow the required number of bi clusters
- A BIGROW benchmark widely accepted over the years and across many different storage systems:
It should take no longer than 10 seconds to write a 100 MB BI file.
UNIX: 16 MB bi clustersize with 1024 16 KB bi blocks
proutil <dbname> -C truncate bi -bi 16384 -biblocksize 16
time -p proutil <dbname> -C bigrow 2 [ -zextendSyncIO ]
Windows: 16 MB bi clustersize with 2048 8 KB bi blocks
proutil <dbname> -C truncate bi -bi 16384 -biblocksize 8
timer.bat "proutil <dbname> -C bigrow 2 [ -zextendSyncIO ]
REM: timer.bat to fake UNIX 'time'
echo %time% < nul
cmd /c %1
echo %time% < nul
When sequential write speed is not optimal and vendors provide metrics to refute results, other tools outside of Progress will provide supporting evidence. One use-case example is recorded in Article: