Feedback
Did this article resolve your question/issue?

   

Article

PASOE Agent crashing without creating core file

« Go Back

Information

 
TitlePASOE Agent crashing without creating core file
URL NamePASOE-Agent-crashing-without-creating-core-file
Article Number000204638
EnvironmentProduct: OpenEdge
Version: 12.2.6
OS: Linux
Question/Problem Description
PASOE agents are crashing intermittently.

Session manager log states the agent is terminated and logs Session level errors:
WARN  [CKsLlaUAT4-mOB_fi9gWQQ-agent-watchdog] - c.p.appserv.PoolMgt.AgentWatchdog - AgentWatchdog(CKsLlaUAT4-mOB_fi9gWQQ) : agent jru8BjMqSG6Y7Ot0_ncYGA PID= 43265 has terminated.
ERROR [thd-4] ROOT:a:00000004 com.progress.appserv.Session - LocalSession(A6B98CCE9D1CB1BCCAF94C91A9FA5DD601E27EE4F66F.pas) : error occurred while reading a message processAgentRsp() = java.net.SocketException: Socket closed:Socket closed. (18300)
WARN  [thd-4] ROOT:a:00000004 c.p.appserv.PoolMgt.Connection - TcpAgentConnectionPool(31lNCxsSRZOA5M1HH5L1NA,5lIOE5a8Ra68dLQTVduxmg) : removeAgentConnection() failed to remove connection : size= 0 max= 1 (18274)
ERROR [thd-4] ROOT:a:00000004 com.progress.appserv.Session - LocalSession(A6B98CCE9D1CB1BCCAF94C91A9FA5DD601E27EE4F66F.pas) : an error occurred while reading response message = java.net.SocketException: Socket closed:Socket closed. (18296)


Agent log just stops logging with no indication the sessions are stopped.
No protrace or core files found on the system. Not even when enabling core file generation (ulimit -c xxxxxx).
Nothing notable written in /var/log/syslog or /var/log/messages.
Steps to Reproduce
Clarifying Information
Error Messageerror occurred while reading a message processAgentRsp() = java.net.SocketException: Socket closed:Socket closed. (18300)
an error occurred while reading response message = java.net.SocketException: Socket closed:Socket closed. (18296)
Defect NumberDefect OCTA-39024
Enhancement Number
Cause
On Unix systems, the PASOE Agent crashes when an unmanaged thread attempts to call a non-thread-safe C runtime function such as rand(). An unmanaged thread is a thread spawned by third party software which has been initiated by an Application call to a shared object function (in a .dll or .so). Threads running in the PASOE Agent should always call thread-safe versions of the C runtime functions; for example, rand_r() instead of rand().
Resolution
Fixed in update 12.2.7 and upcoming release 12.5
Workaround
Notes
The reason no procore file was created (and no syslog() entry logged) is likely to be partly related to defect OCTA-38702 which caused a crash of a worker thread.
If a secondary crash occurred on the same thread while the signal handler was processing the first crash, then the OS shoots the process in the head (without a trace).
Ultimately it was found that the process was running a third-party library (libzmq.so.5) that was spinning up its own thread and masking signals.
Last Modified Date11/9/2021 11:10 AM
Attachment 
Files
Disclaimer The origins of the information on this site may be internal or external to Progress Software Corporation (“Progress”). Progress Software Corporation makes all reasonable efforts to verify this information. However, the information provided is for your information only. Progress Software Corporation makes no explicit or implied claims to the validity of this information.

Any sample code provided on this site is not supported under any Progress support program or service. The sample code is provided on an "AS IS" basis. Progress makes no warranties, express or implied, and disclaims all implied warranties including, without limitation, the implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample code is borne by the user. In no event shall Progress, its employees, or anyone else involved in the creation, production, or delivery of the code be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample code, even if Progress has been advised of the possibility of such damages.