Feedback
Did this article resolve your question/issue?

   

Article

Does Progress use Synchronous or Asynchronous I/O?

« Go Back

Information

 
TitleDoes Progress use Synchronous or Asynchronous I/O?
URL NameP108815
Article Number000149598
EnvironmentProduct: Progress OpenEdge
Version: All supported versions
OS: All supported platforms
Question/Problem Description
Does Progress use Synchronous or Asynchronous I/O?
Does Progress use  buffered or unbuffered I/O?
Does Progress use aio_read() or aio_write() system calls?
Steps to Reproduce
Clarifying Information
Error Message
Defect Number
Enhancement Number
Cause
Resolution
Synchronous I/O (direct I/O) means wait for each I/O to complete.

Asynchronous I/O means issuing I/O then wait for them to complete or issuing an I/O without waiting and being allowed to perform other processing and then ask or wait for the previously issued I/O completion. There are ways for one thread to wait on another thread's I/O completion. The way the Asynchronous I/O is performed is platform dependent.

Progress does not do any Asynchronous I/O per se, as aio_read() or aio_write() system calls are not used. Progress implements its own Asynchronous I/O "processing" through its file (ai, bi and db) buffering and I/O by page writers. While the process that makes a change to a block in a buffer does not actually perform the I/O, it continues its processing while the I/O is waiting in a buffer pool (-aibufs, -bibufs or -B) to be written out by another process (the AIW, BIW, APW).

Progress uses two flavors of synchronous I/O:

  1. Buffered Synchronous I/O and 
  2. Unbuffered Synchronous I/O (direct I/O). Some consider buffered I/O to be Asynchronous but it truly is not when Asynchronous I/O is defined as allowing other operations to execute by the same thread before an issued I/O completes
To guarantee crash recovery, a write ahead logging (WAL) rule is enforced which basically states that we will always write to the bi file first and then to the data files. To follow this protocol, not only do we need to issue the writes in the proper order, but we must ensure that the physical changes to the disk are done in the proper order.

Therefore, the current implementation of Progress I/O is as follows:

1. Unbuffered synchronous I/O (O_RDWR|O_DSYNC) to the bi, crash recovery guaranteed unless running with -i (no-integrity) -R (non-raw) startup flags:
  • When running with the -R (non-raw) startup flag - we do Buffered Synchronous I/O (O_RDWR) to the bi file. This allows rollback and possibly crash recovery. If the database were to crash, crash recovery should work as all the data needed should still be in the file system's cache if it hasn't made it to physical disk yet. However, if the system were to crash, it is not guaranteed that crash recovery will work - it most likely will not.
  • When running with the -i (no-integrity) startup flag (-i forces -R) - we do Buffered Synchronous I/O (O_RDWR) to the bi file and no bi/data order synchronization. This means that rollback should work, but if the database were to crash or a user get killed even if the system were not to crash, there is no guarantee that crash recovery will complete. In this case, you are prevented from connecting to the database - we don't even try to recover.
** The last session was run with the no integrity (-i) parameter. (509) 
** Your database cannot be repaired. You must restore a backup copy. (510)

2.   When running with -directio startup flag - we do Unbuffered Synchronous I/O (O_RDWR|O_DSYNC) to the data files. This means direct writes to disk for data files and allows us to avoid needing fdatasync() calls at checkpoints. Some customers see improvements running this way, some don't. It all depends on the application. The idea is that since Progress maintains a Buffer Pool, why do we need to use the file system's "buffer pool" as well? This would mean we have a cache on top of a cache. There are cases where double caching is better but that discussion is not the scope of this Article.

3.   When not running with -directio - we do Buffered Synchronous I/O (O_RDWR) to the datafiles which means that we must issue an fdatasync() call when we want to guarantee that the data I/O has made it out of the file system's cache and has been written to disk (usually only at checkpoint time).

The difference between Buffered I/O and Unbuffered Synchronous I/O is that an Unbuffered Synchronous I/O is guaranteed to be on disk when returned from the write() system call. A Buffered Synchronous I/O is guaranteed to exist, either in the file system's cache or possibly on disk when returned from the write() system call.

In the above discussion, fdatasync() on UNIX, FileFlushBuffers() is used on Windows
Workaround
Notes
Last Modified Date11/20/2020 7:37 AM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.