Feedback
Did this article resolve your question/issue?

   

Article

proGetStack / kill -SIGUSR1 causes memory leak, eventually crash in AppServer

Information

 
TitleproGetStack / kill -SIGUSR1 causes memory leak, eventually crash in AppServer
URL Name000035325
Article Number000159908
EnvironmentProduct: OpenEdge
Version: 10.x, 11.x
OS: Linux
Question/Problem Description
OpenEdge clients leak memory if sent a SIGUSR1 signal to create protrace file
Affects all clients, including AppServer / WebSpeed agents

Also affects proGetStack script - on Linux that script just sends the SIGUSR1 signal to a PID.
Also affects other tools that rely on proGetStack script (adminServer commands such as 'asbman -agentdetail' , querying the appserver from Progress Developer Studio.)

If signal is sent often enough to same process, it crashes with SYSTEM ERROR: Memory violation. (49)
Steps to Reproduce
Clarifying Information
Key identifying signs of this issue:

Mapped memory regions using the agent's PID before and after running proGetStack on the agent PID, shows memory maps are not unmapped after calling proGetStack:
$   sudo cat /proc/[pid]/maps
mapped memory regions of the agents PID's grow eventually exhausting the mmap region causing the agent process terminate.

strace shows for every open('library') call there are corresponding mmap, munmap and close, execptions being /usr/lib64/libstdc++.so.6 and /lib64/libgcc_s.so.1 whose memory mappings aren't being unmapped.

Method of generating protrace files varies between operating systems; proGetStack script is a tool to make that operation platform-independent.
 
Error MessageSYSTEM ERROR: Memory violation. (49)
Defect NumberDefect PSC00248179 / OE00225436
Enhancement Number
Cause
Some libraries will not be unmapped before closing the respective files.

Signal handler for SIGUSR1 initiating protrace file creation.

To create that stack trace the program calls 'backtrace' OS function, gets call stack from that and tries to map those addresses into program or external library symbol names.  In order to accomplish that, the program opens the dynamic libraries, mmaps them and then unmaps and closes them.
In some cases the unmapping of the libraries fails.
Resolution
Upgrade to OpenEdge 11.6.3, 11.7 or later where library files that do not have a symbol table are unmapped properly.
Workaround
Avoid sending -SIGUSR1 signals when possible.
If needed, comment out the 'kill' command within the DLC/bin/proGetStack.sh script.
Notes
Last Modified Date1/23/2019 1:05 PM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.