Data Execution Prevention (DEP) is a set of hardware and software technologies that perform additional checks on memory to help prevent malicious code from running on a system.
The primary benefit of DEP is to help prevent code execution from data pages.
Typically, code is not executed from the default heap and the stack.
The DEP feature was first included in Windows XP SP2, Window Server 2003 SP1, and above.
DEP is enforced by hardware and by software:
- Hardware-enforced DEP detects code that is running from these locations and raises an exception when execution occurs.
- Software-enforced DEP can help prevent malicious code from taking advantage of exception-handling mechanisms in Windows.
If DEP is set to "Turn on DEP for all programs and services except those I select:",
various behaviors can occur in the OpenEdge environment such as:
- An OpenEdge database not starting,
- Not being able to pull data into Microsoft SQL Server from a Progress database,
- Similar problems with the OpenEdge client process prowin32.exe if it invokes ActiveX controls or other Windows programs and utilities, or
- The OpenEdge AdminServer not starting, where much of the problem stems from the fact that the associated Java process "java.exe" is being prevented from running in memory with the current configured DEP settings.
To correct this behavior, change Windows' DEP setting to "Turn on DEP for essential Windows programs and services only",
or add exceptions for the related OpenEdge installed executable such as java.exe, prowin32.exe etc.
To change the DEP setting:
1. Navigate to My Computer > Properties > Performance > Settings > Data Execution Prevention.
2. Do one of the following:
- Select "Turn on DEP for essential Windows programs and services only"
- Select "Turn on DEP for all programs and services except those I select", and click Add to add the programs that should not use the DEP feature (typically %DLC%\jre\bin\java.exe and/or %DLC%\bin\prowin32.exe).
3. Reboot the machine.
NOTE: OpenEdge became DEP compliant in all releases starting with Release 11.0, by compiling with Visual Studio 2010 or later using /NXCOMPAT:YES.