Reputation: 432
I'm perplexed by the above, which keeps cropping up intermittently whenever I execute a stored proc within a package on our new Oracle installation.
The sproc is called from WCF, which returns the following exception :
<ExceptionDetail xmlns="http://schemas.datacontract.org/2004/07/System.ServiceModel" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<HelpLink i:nil="true"/>
<InnerException>
<HelpLink i:nil="true"/>
<InnerException i:nil="true"/>
<Message>ORA-12537: Network Session: End of file</Message>
<StackTrace><![CDATA[at OracleInternal.Network.ReaderStream.Read(OraBuf OB)
at OracleInternal.TTC.OraBufReader.GetDataFromNetwork()
at OracleInternal.TTC.OraBufReader.Read(Boolean bIgnoreData)
at OracleInternal.TTC.MarshallingEngine.UnmarshalUB1(Boolean bIgnoreData)
at OracleInternal.TTC.TTCExecuteSql.ReceiveExecuteResponse(Accessor[]& defineAccessors, Accessor[] bindAccessors, Boolean bHasReturningParams, SQLMetaData& sqlMetaData, SqlStatementType statementType, Int64 noOfRowsFetchedLastTime, Int32 noOfRowsToFetch, Int32& noOfRowsFetched, Int64& queryId, Int32 longFetchSize, Int64 initialLOBFetchSize, Int64[] scnFromExecution, Boolean bAllInputBinds, Int32 arrayBindCount, DataUnmarshaller& dataUnmarshaller, MarshalBindParameterValueHelper& marshalBindParamsHelper, Int64[]& rowsAffectedByArrayBind, Boolean bDefineDone, Boolean& bMoreThanOneRowAffectedByDmlWithRetClause, List`1& implicitRSList, Boolean bLOBArrayFetchRequired)]]></StackTrace>
<Type>OracleInternal.Network.NetworkException</Type>
</InnerException>
<Message>ORA-12537: Network Session: End of file</Message>
<StackTrace><![CDATA[at Oracle.ManagedDataAccess.Client.OracleException.HandleError(OracleTraceLevel level, OracleTraceTag tag, Exception ex, OracleLogicalTransaction oracleLogicalTransaction)
at OracleInternal.TTC.TTCExecuteSql.ReceiveExecuteResponse(Accessor[]& defineAccessors, Accessor[] bindAccessors, Boolean bHasReturningParams, SQLMetaData& sqlMetaData, SqlStatementType statementType, Int64 noOfRowsFetchedLastTime, Int32 noOfRowsToFetch, Int32& noOfRowsFetched, Int64& queryId, Int32 longFetchSize, Int64 initialLOBFetchSize, Int64[] scnFromExecution, Boolean bAllInputBinds, Int32 arrayBindCount, DataUnmarshaller& dataUnmarshaller, MarshalBindParameterValueHelper& marshalBindParamsHelper, Int64[]& rowsAffectedByArrayBind, Boolean bDefineDone, Boolean& bMoreThanOneRowAffectedByDmlWithRetClause, List`1& implicitRSList, Boolean bLOBArrayFetchRequired)
at OracleInternal.ServiceObjects.OracleDataReaderImpl.FetchMoreRows(Int32 noOfRowsToFetch, Boolean fillReader, Boolean returnPSTypes)
at Oracle.ManagedDataAccess.Client.OracleDataReader.Read()
at System.Data.Common.DataAdapter.FillLoadDataRow(SchemaMapping mapping)
at System.Data.Common.DataAdapter.FillFromReader(DataSet dataset, DataTable datatable, String srcTable, DataReaderContainer dataReader, Int32 startRecord, Int32 maxRecords, DataColumn parentChapterColumn, Object parentChapterValue)
at System.Data.Common.DataAdapter.Fill(DataSet dataSet, String srcTable, IDataReader dataReader, Int32 startRecord, Int32 maxRecords)
at Oracle.ManagedDataAccess.Client.OracleDataAdapter.Fill(DataSet dataSet, String srcTable, IDataReader dataReader, Int32 startRecord, Int32 maxRecords)
at Oracle.ManagedDataAccess.Client.OracleDataAdapter.Fill(DataSet dataSet, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet)
at ***REDACTED***
at ***REDACTED***
at ***REDACTED***
at SyncInvokeQtyAdjustments_Get(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage41(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)]]></StackTrace>
<Type>Oracle.ManagedDataAccess.Client.OracleException</Type>
</ExceptionDetail>
There doesn't seem to be a pattern to it either - sometimes it works fine, sometimes it doesn't.
Here's stuff I've tried :
Ensured disposal of objects in the generic code. Here's the test code currently used :
using System.Data;
using System.Data.Common;
using Oracle.ManagedDataAccess.Client;
/// <summary>
/// Extensions for xxxxx.
/// </summary>
/// <seealso cref="System.Data.Entity.DbContext" />
public partial class xxxxEntities
{
public virtual DataSet ExecuteStoredProcedureAsDataSet(string storedProcName, int commandTimeoutSeconds, object[] parameters)
{
var sqlCommand = this.Database.Connection.CreateCommand();
var resultSet = new DataSet();
sqlCommand.CommandType = CommandType.StoredProcedure;
sqlCommand.CommandTimeout = commandTimeoutSeconds;
sqlCommand.CommandText = storedProcName;
sqlCommand.Parameters.AddRange(parameters);
var referenceCursor = new OracleParameter("P_DATAOUT", OracleDbType.RefCursor)
{
Direction = ParameterDirection.Output
};
sqlCommand.Parameters.Add(referenceCursor);
using (var adapter = CreateDataAdapter(this.Database.Connection))
{
adapter.SelectCommand = sqlCommand;
adapter.Fill(resultSet);
}
referenceCursor.Dispose();
sqlCommand.Dispose();
return resultSet;
}
/// <summary>
/// Creates the data adapter.
/// </summary>
/// <param name="connection">The connection.</param>
/// <returns>DbDataAdapter.</returns>
DbDataAdapter CreateDataAdapter(DbConnection connection)
{
return DbProviderFactories.GetFactory(connection).CreateDataAdapter();
}
}
Cutting down the query in the stored proc to 'fixed' values - to make sure it was nothing in the sproc causing the issue. I did find some interesting problems described below.
So, I'm at a loss. I turned on ODP tracing - and did notice it seems to be having issues when dealing with the pool. The first time, I got this :
2016-07-26 11:43:55.268765 TID:6 (PRI) (SVC) (ENT) OracleConnectionImpl.EvaluateDbMajorMinorPatchsetVersion()
2016-07-26 11:43:55.268765 TID:6 (PRI) (SVC) (EXT) OracleConnectionImpl.EvaluateDbMajorMinorPatchsetVersion()
2016-07-26 11:43:55.268765 TID:6 (PRI) (SVC) (EXT) OracleConnectionImpl.Connect() (oper=open) (aff=n/a) (inst=WMSQA) (affmatch=n/a) (sessid=2540:42190) (F;F;F;WMSQA;N) (pmid=29382239)
2016-07-26 11:43:55.268765 TID:6 (PRI) (ENT) (CP) PoolManager`3.PutNewPR() (aff=n/a) (inst=WMSQA) (affmatch=n/a) (sessid=2540:42190) (F;F;F;WMSQA;N) (pmid=29382239)
2016-07-26 11:43:55.268765 TID:6 (PRI) (ENT) (CP) OraclePool.PutNewPR()
2016-07-26 11:43:55.268765 TID:6 (PRI) (ENT) (CP) Pool`3.PutNewPR()
2016-07-26 11:43:55.268765 TID:6 (PRI) (CP) Pool`3.PutNewPR() (oper=cp:put:reg1) (aff=n/a) (inst=WMSQA) (affmatch=n/a) (sessid=2540:42190) (F;F;T;WMSQA;N) (pmid=29382239)
2016-07-26 11:43:55.268765 TID:6 (PRI) (EXT) (CP) Pool`3.PutNewPR()
2016-07-26 11:43:55.268765 TID:6 (PRI) (EXT) (CP) OraclePool.PutNewPR()
2016-07-26 11:43:55.268765 TID:6 (PRI) (EXT) (CP) PoolManager`3.PutNewPR() (aff=n/a) (inst=WMSQA) (affmatch=n/a) (sessid=2540:42190) (F;F;T;WMSQA;N) (pmid=29382239)
2016-07-26 11:43:55.269765 TID:6 (PRI) (EXT) (CP) PoolManager`3.CreateNewPR() (aff=n/a) (inst=WMSQA) (affmatch=n/a) (sessid=2540:42190) (F;F;T;WMSQA;N) (pmid=29382239)
2016-07-26 11:43:55.269765 TID:6 (PRI) (CP) PoolManager`3.PopulatePool() PopulatePoolThreadFunc(created=2; max=25; total=3)
2016-07-26 11:43:55.269765 TID:6 (PRI) (EXT) (CP) PoolManager`3.PopulatePool()
2016-07-26 11:43:55.880826 TID:15 (PRI) (ENT) TTCExecuteSql.ReceiveExecuteResponse()
2016-07-26 11:43:55.881826 TID:15 (PRI) (ENT) OracleException.ctor()
2016-07-26 11:43:55.885826 TID:15 (PRI) (ENT) OracleError.ctor()
2016-07-26 11:43:55.886826 TID:15 (PRI) (EXT) OracleError.ctor()
2016-07-26 11:43:55.886826 TID:15 (PRI) (EXT) OracleException.ctor()
2016-07-26 11:43:55.890827 TID:15 (PRI) (TTC) (ERR) TTCExecuteSql.ReceiveExecuteResponse() (txnid=n/a) OracleInternal.Network.NetworkException (0x000030F9): ORA-12537: Network Session: End of file
at OracleInternal.Network.ReaderStream.Read(OraBuf OB)
at OracleInternal.TTC.OraBufReader.GetDataFromNetwork()
at OracleInternal.TTC.OraBufReader.Read(Boolean bIgnoreData)
at OracleInternal.TTC.MarshallingEngine.UnmarshalUB1(Boolean bIgnoreData)
at OracleInternal.TTC.TTCExecuteSql.ReceiveExecuteResponse(Accessor[]& defineAccessors, Accessor[] bindAccessors, Boolean bHasReturningParams, SQLMetaData& sqlMetaData, SqlStatementType statementType, Int64 noOfRowsFetchedLastTime, Int32 noOfRowsToFetch, Int32& noOfRowsFetched, Int64& queryId, Int32 longFetchSize, Int64 initialLOBFetchSize, Int64[] scnFromExecution, Boolean bAllInputBinds, Int32 arrayBindCount, DataUnmarshaller& dataUnmarshaller, MarshalBindParameterValueHelper& marshalBindParamsHelper, Int64[]& rowsAffectedByArrayBind, Boolean bDefineDone, Boolean& bMoreThanOneRowAffectedByDmlWithRetClause, List`1& implicitRSList, Boolean bLOBArrayFetchRequired)
Ramping up the tracing to 127, I just got this :
2016-07-26 14:22:11.459348 TID:1 (PRI) (SVC) (ENT) OracleDataReaderImpl.FetchMoreRows()
2016-07-26 14:22:11.459348 TID:1 (PRI) (SVC) (ENT) OracleConnectionImpl.AddAllPiggyBackRequests()
2016-07-26 14:22:11.459348 TID:1 (PRI) (TTC) (ENT) TTCClose.Write()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCFunction.WriteFunctionHeader()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCMessage.WriteTTCCode()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (EXT) TTCMessage.WriteTTCCode()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (EXT) TTCFunction.WriteFunctionHeader()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (EXT) TTCClose.Write()
2016-07-26 14:22:11.460348 TID:1 (PRI) (SVC) (EXT) OracleConnectionImpl.AddAllPiggyBackRequests()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCExecuteSql.SendExecuteRequest()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCExecuteSql.GetExecuteOptions()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (EXT) TTCExecuteSql.GetExecuteOptions()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCExecuteSql.WriteOall8Message()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCFunction.WriteFunctionHeader()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCMessage.WriteTTCCode()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (EXT) TTCMessage.WriteTTCCode()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (EXT) TTCFunction.WriteFunctionHeader()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCExecuteSql.WritePisdef()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (EXT) TTCExecuteSql.WritePisdef()
2016-07-26 14:22:11.460348 TID:1 (PRI) (TTC) (ENT) TTCExecuteSql.WritePisdefData()
2016-07-26 14:22:11.461348 TID:1 (PRI) (TTC) (EXT) TTCExecuteSql.WritePisdefData()
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 00 4C 00 00 06 00 00 00 |.L......|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 00 00 |.. |
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 11 69 00 01 01 01 01 02 |.i......|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 03 5E 00 02 80 40 01 03 |.^...@..|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 00 00 01 01 0D 00 00 00 |........|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 00 01 01 00 00 00 00 00 |........|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 00 00 00 00 00 00 01 00 |........|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 00 00 00 00 00 00 00 00 |........|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 02 01 36 00 00 00 03 A7 |..6.....|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) EB C6 00 01 01 00 00 00 |........|
2016-07-26 14:22:11.461348 TID:1 (NET) (SND) 00 00 |.. |
2016-07-26 14:22:11.461348 TID:1 (PRI) (TTC) (EXT) TTCExecuteSql.WriteOall8Message()
2016-07-26 14:22:11.461348 TID:1 (PRI) (TTC) (EXT) TTCExecuteSql.SendExecuteRequest()
2016-07-26 14:22:11.461348 TID:1 (PRI) (TTC) (ENT) TTCExecuteSql.ReceiveExecuteResponse()
2016-07-26 14:22:11.461348 TID:1 (PRI) (BUF) (COBP.GET) (poolid:2) (key:8192) (bufid:8) (count:5) (OraBufReader.GetDataFromNetwork)
2016-07-26 14:22:12.161348 TID:1 (PRI) (ENT) TTCExecuteSql.ReceiveExecuteResponse()
2016-07-26 14:22:12.162348 TID:1 (PRI) (ENT) OracleException.ctor()
2016-07-26 14:22:12.162348 TID:1 (PRI) (ENT) OracleError.ctor()
2016-07-26 14:22:12.162348 TID:1 (PRI) (EXT) OracleError.ctor()
2016-07-26 14:22:12.162348 TID:1 (PRI) (EXT) OracleException.ctor()
2016-07-26 14:22:12.163348 TID:1 (PRI) (TTC) (ERR) TTCExecuteSql.ReceiveExecuteResponse() (txnid=n/a) OracleInternal.Network.NetworkException (0x000030F9): ORA-12537: Network Session: End of file
at OracleInternal.Network.ReaderStream.Read(OraBuf OB)
at OracleInternal.TTC.OraBufReader.GetDataFromNetwork()
at OracleInternal.TTC.OraBufReader.Read(Boolean bIgnoreData)
at OracleInternal.TTC.MarshallingEngine.UnmarshalUB1(Boolean bIgnoreData)
at OracleInternal.TTC.TTCExecuteSql.ReceiveExecuteResponse(Accessor[]& defineAccessors, Accessor[] bindAccessors, Boolean bHasReturningParams, SQLMetaData& sqlMetaData, SqlStatementType statementType, Int64 noOfRowsFetchedLastTime, Int32 noOfRowsToFetch, Int32& noOfRowsFetched, Int64& queryId, Int32 longFetchSize, Int64 initialLOBFetchSize, Int64[] scnFromExecution, Boolean bAllInputBinds, Int32 arrayBindCount, DataUnmarshaller& dataUnmarshaller, MarshalBindParameterValueHelper& marshalBindParamsHelper, Int64[]& rowsAffectedByArrayBind, Boolean bDefineDone, Boolean& bMoreThanOneRowAffectedByDmlWithRetClause, List`1& implicitRSList, Boolean bLOBArrayFetchRequired)
2016-07-26 14:22:12.163348 TID:1 (PRI) (EXT) TTCExecuteSql.ReceiveExecuteResponse()
2016-07-26 14:22:12.163348 TID:1 (PRI) (TTC) (EXT) TTCExecuteSql.ReceiveExecuteResponse()
2016-07-26 14:22:12.163348 TID:1 (PRI) (ENT) OracleDataReaderImpl.FetchMoreRows()
2016-07-26 14:22:12.163348 TID:1 (PRI) (SVC) (ERR) OracleDataReaderImpl.FetchMoreRows() (txnid=n/a) Oracle.ManagedDataAccess.Client.OracleException (0x80004005): ORA-12537: Network Session: End of file ---> OracleInternal.Network.NetworkException (0x000030F9): ORA-12537: Network Session: End of file
I tweaked the query in the sproc by commenting out the select statements and where clauses, and gradually introduced them back in to see if it's the query that's the problem (running it in Oracle Developer works fine, though). I've narrowed it down to this statement - if I reintroduce it, I get the End of file exceptions again :
WHERE
-- Only handle records that contain a numeric value in the ref field.
ISNUMERIC(PTT.REF_FIELD_2) = 1
This is what the ISNUMERIC function looks like in the package ;
FUNCTION ISNUMERIC (p_string IN VARCHAR2) RETURN INT
IS
v_new_num NUMBER;
BEGIN
IF p_string IS NULL
THEN
RETURN 0;
END IF;
v_new_num := TO_NUMBER(p_string);
RETURN 1;
EXCEPTION
WHEN VALUE_ERROR THEN
RETURN 0;
END ISNUMERIC;
At first I thought it might be the number of exceptions caught was preventing further execution due to the number of exceptions. However I simplified the function to just return '1' and it still didn't work every time.
The results of the query are passed back via a ref cursor e.g.
PROCEDURE DEVSP_API_GETxxxx(
P_STARTDATE IN DATE
, P_ENDDATE IN DATE
, P_xxx IN P_xxx_TYPE
, <snip>
, P_DATAOUT OUT sys_refcursor
) AS
BEGIN
OPEN P_DATAOUT FOR
SELECT
'asdsa' as "x",
'sadasdsad' as "z",
0 as "Qty",
<snip>
Finally, I removed all Function calls from the select statements and the where clauses - and that instantly stops the error from occurring - I replaced them with the same logic the function would run - just inline within the SELECT and WHERE statements.
So what's going on? Is there a limitation in ODP.net when running sprocs within a package that contain calls to functions? Seems a bit odd.
The workaround is to just not use functions, but I don't want to do that as some select statements have conditional expressions in them which repeat the same logic in Union-based queries.
Any help or pointers much appreciated!
Ps. for transparency, I posted this on the Oracle forum last week but have had no responses.
Update : 03-Aug-2016
Finally managed to get access to the server to try a bit of tracing. Here's what came up in the trace log when the process ends prematurely :
*** 2016-08-03 10:14:39.111
*** SESSION ID:(2526.53330) 2016-08-03 10:14:39.111
*** CLIENT ID:() 2016-08-03 10:14:39.111
*** SERVICE NAME:(XXXXX) 2016-08-03 10:14:39.111
*** MODULE NAME:(iisexpress.exe) 2016-08-03 10:14:39.111
*** CLIENT DRIVER:(ODPM.NET) 2016-08-03 10:14:39.111
*** ACTION NAME:() 2016-08-03 10:14:39.111
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x0] [PC:0x3CEBE1D, pmucfst()+13] [flags: 0x0, count: 1]
Incident 109041 created, dump file /u01/app/oracle/diag/rdbms/xxxx/XXXX/incident/incdir_109041/XXXXX_ora_13149_i109041.trc
ORA-07445: exception encountered: core dump [pmucfst()+13] [SIGSEGV] [ADDR:0x0] [PC:0x3CEBE1D] [Address not mapped to object] []
ssexhd: crashing the process...
Shadow_Core_Dump = PARTIAL
I can also see a core dump in the same folder. I've tried grepping the bucket trace files in there to see if there's anything obvious - but to be honest it's difficult to search for something when you don't know what you're looking for!
I can see a shedload of core dumps from previous attempts to run the sprocs whenever it used a function - so it's at least reasonably consistent.
Upvotes: 1
Views: 4141
Reputation: 11
I ran into the same issue. Oracle Support responded with:
It looks like you are hitting Bug 18191823 - Hang / ORA-600 [19708] etc.. referencing stale REFCURSOR bind ( Doc ID 18191823.8 )
Workaround
Ensure the duration of the bound PL/SQL variable is at least as long as that of the ref cursor to which it is bound.
The fix is first included in July 2016 patches.
Upvotes: 1