Considerations when enabling database large file support
The large files feature was added to the Progress 9.1C Enterprise Database license to enable database extents to grow larger than 2GB. Large files are not supported and cannot be enabled in prior versions.
Only an Enterprise database license supports the largefiles feature. Databases that have the large file feature enabled cannot be accessed by previous versions which don't have the code necessary to use large files or environments without an Enterprise Database license installed.
Prior to Progress 9.1E, older Linux OS versions are still supported that do not have large file support, therefore large files support could not be enabled for the Linux port. Since Progress 9.1E large files are supported on certified on Linux versions.
Progress does not support files larger than 2 Gigabytes on SCO OpenServer or SCO UnixWare. The port for SCO UnixWare is the same as for SCO OpenServer; because SCO OpenServer does not support large files, large files support is not enabled for the SCO UnixWare port either.
Before enabling the database largefiles feature, ensure the file system and user limits/quotas are without without 2GB limits
- The file system(s) where the database resides must be enabled to support files larger than 2GB. Some operating systems, file systems do not support large files by default and large files support must be enabled explicitly.
- On an OS level, ensure that the ulimit for users that access the database is set to 'unlimited', or a limit higher than the expected size of the database files.
- Various Operating Systems have differing limits on the maximum file sizes they support. For example, the maximum size of files on IBM AIX is 64Gb.
Large file processing is not enabled by default. For databases that do not have large file processing enabled, extents will be restricted to a maximum of 2GB, or the size specified for it in a .st file, whichever is less.
Once large file processing is enabled for a database:
Which files does enablelargefiles affect?
All database files will be allowed to exceed 2GB in size when large file processing is enabled:
- After Image Extents (.an)
- Before Image Extents (.bn)
- Data Extents (.dn)
- Transaction Log File (.tn)
Variable length extents will be allowed to expand to a maximum of 1 TB depending on the database blocksize and records per block.
Fixed (f) and fixed-variable (v) length extents with specified maximum sizes beyond 2GB are allowed. Specifying a maximum size for a variable length extent does not cause it to be pre-allocated to that size. It only means that the extent will not be allowed to grow beyond the specified size. At most one variable length extent per Storage Area is allowed and it must be the last extent in the area
The following lines illustrate definition of two variable length extents, one with a maximum size of 3GB and one that may expand to the maximum area size, assuming large file processing is enabled.
d "Employee":7,32;512 /usr1/emp v 3145728
d "Invoice":8,32;8 /usr1/inv
Enabling Database large files has no relation to other files.
- Progress backup(PROBKUP), restore(PROREST), binary dump, and binary load were enhanced to support large files in Progress 9.1B (except for Linux, where this feature has been introduced in OpenEdge 10).
- Prior to OpenEdge 10.1C, 2GB limits still apply to other files, in particular those directly accessed by the 4GL
Data Dictionary. d files prevent either and or 2GB dump loads depending on the Progress version and operating system.
How to enable large file processing for a database: 1. PROSTRCT CREATE
Files mentioned in the ABL statements INPUT FROM and OUTPUT TO are limited to 2 gigabytes in length.
Client Temp files such as lbi and srt temporary files for local before-imaging and sorting/result-listing are subject to the 2-gigabyte limit.
Refer to the Database Administration Guide and Reference, Progress Database Limits
When creating a new Enterprise database, if the .st file specifies a fixed length extent size (f) or a maximum size for a fixed-variable length extent (v) that is greater than 2GB, large file processing will be enabled for that database.
Example: The following command will create a database with large file processing enabled:
$ prostrct create largedb largedb.st
Where the largedb.st structure file contains the following:
# largedb.st to create database largedb with large file processing enabled.
2. PROUTIL -C EnableLargeFiles
# a fixed length bi file of 1GB and
# a fixed-variable length bi file with a maximum size of 4GB
b tests/largedb.b1 f 1048576
b tests/largedb.b2 v 4194304
# SCHEMA AREA with a fixed length file of 3GB and
# a fixed-variable length file with a maximum size of 3GB
d "Schema Area":6,64;1 tests/largedb.d1 f 3145728
d "Schema Area":6,64;1 tests/largedb.d2 v 3145728
# TABLE AREA with a fixed length file of just over 2GB and
# a variable length file with a maximum size of 1TB
d "Table Area":7,64 ;512 tests/largedb_7.d1 f 2097280
d "Table Area":7,64;512 tests/largedb_7.d2
# a fixed length ai file of 2GB
# and a variable length file with a maximum size of 1TB
a tests/largedb.a1 f 2097152
Large file processing can be enabled for an Enterprise database by running the PROUTIL EnableLargeFiles option. The syntax of the command is as follows:
$ proutil <dbname> -C EnableLargeFiles
Where "<dbname>" is the name of the database.
- The database must be offline when enabling large file processing.
- There is no corresponding command to disable large file processing for a database.
- Once large file processing is enabled for a database, the database cannot be accessed by Progress versions prior to 9.1C.
- In Progress versions 9.1E and later, the proutil EnableLargeFiles utility may be used to enable large file processing on a void database
In Progress versions prior to 9.1E, if a database is created using PROCOPY or PROREST utilities and the source database has large file processing enabled, large file processing will be enabled for the target database. Similarly, if a database with large file processing enabled is copied to a void Progress database, the target will have large files enabled. Copying of a database without large file processing to another that has large files enabled will not cancel large files on the target.
In Progress versions 9.1E and later, if the target database of a PROCOPY or PROREST operation exists (even if only as a void database), the target’s large files setting will not be changed. Void databases in these versions may be enabled for large file processing using the PROUTIL EnableLargeFiles utility. This change was made to allow the designer of a database to determine its large file status, rather than the PROCOPY or PROREST function.
As in Progress versions prior to 9.1E, if the target database in a PROCOPY or PROREST operation does not exist, the resulting database’s large file processing state is set to match that of the source database.
The PROSTRCT ADD utility used to add extents to an existing database, does not enable large file processing. To add a file with a specified size larger than 2GB, large file processing must already be enabled for the database as described above.
Enabling large file processing for a database will allow existing variable length extents that were created without a specified maximum size to expand beyond 2GB. New fixed and variable length extents with maximum sizes greater than 2GB may be added. Existing extents with a maximum size that was specified in a .st file will not be affected.