Feedback
Did this article resolve your question/issue?

   

Article

How to produce a stack trace for a running OpenEdge process without killing it

Information

 
TitleHow to produce a stack trace for a running OpenEdge process without killing it
URL NameP112486
Article Number000184833
EnvironmentProduct: Progress
Version: 9.x
Product: OpenEdge
Version: 10.x, 11.x
OS: UNIX
Question/Problem Description
How to produce a stack trace for a running process without killing it ?
Which signal should be used to produce a stack trace for a running process without terminating the process?
How to generate a protrace C stacktrace information against a hanging OpenEdge process?
Steps to Reproduce
Clarifying Information
Error Message
Defect Number
Enhancement Number
Cause
Resolution
Under UNIX, a running OpenEdge process (_progress, _proapsv or _mprosrv) can be sent a signal so that it will generate a C stack trace and then resume execution.
 
This is particularly useful under runaway process conditions in order to find exactly in which functions the process is spinning by sending several signals in succession.  Information can also be used to understand what a "hanging" process is actually waiting for.  Example: a message from the network or a user-interface interaction.
 
Any signal that is sent to a specific process can only be carried out by the root user or a user that belongs to the same UNIX Group as the user under which the process is running if the suid bit is appropriately set on the executable.

Ensure that the Operating System is configured to generate system dumps.  
  • ulimit -a can be used to identify if core size has been set.
  • ulimit -c  can be used to set the core file size to unlimited or some value like 524288.
SIGUSR1 is a special signal that causes the process to dump a procore and protrace.pid file, but does not have the process exit.
This is a OpenEdge feature. Non-OpenEdge executables will react differently, if at all, to signal SIGUSR1.

The following syntax as root where ulimit -c is not disabled and there are no suid or hardlimits preventing core file generation, will produce a file named "protrace.<PID>" in the directory where the process was started from, containing a readable stack trace: 
 
$  kill -SIGUSR1 <PID>
 
Before sending the signal:
  • Ensure the PID exists with ps -ef | grep <pid> for example.
  • Do not pass 1 as the PID, -1 is the “init”process.
  • Make sure the PID has no children – this is often overlooked.
  • SIGUSR1 is not trapped in the shell executing the cmd. If trap is set at the shell level the OpenEdge process will never receive the signal. The UNIX trap command first needs to be removed from scripts used to invoke _progres or any other OpenEdge executables. Type "trap" at the command line to confirm if the shell has any traps set.
SIGUSR1 translates to the following signals on the following Operating Systems:
 
Operating       Signal
System          Number
--------------- ------
AIX              30(*)
HP-UX            16
HP Tru64         30
RedHat Linux     10
Solaris          16
UnixWare         N/A
 
(*) Under AIX this feature is only available starting with OpenEdge 10.1A; in previous releases, Progress executable's are built to simply ignore SIGUSR1.
 
For example:
  • Under RedHat Linux, issuing kill -SIGUSR1 is the same as issuing kill -10.
  • Under Solaris, SIGUSR1 maps to signal 16, but the stack traces obtained by issuing kill -16 are of no use.
Since OpenEdge 10.1C further enhancements were introduced to protrace output by providing an ABL stack trace with a list of persistent procedures or classes that are in memory and their handle ID's along with the C stack trace for any ABL client process (which includes the WebSpeed and AppServer agents). This Enhancement PSC00183237/OE00147006 is available on both UNIX and WINDOWS platforms. It is advisable to upgrade to OpenEdge 11.6.4, 11.7.0 or later when using this feature on Linux. For further details refer to Articles:
Workaround
Notes
Last Modified Date11/20/2020 7:26 AM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.