Today I ran the 2111 Update and I have found that all of my secondary sites are going from 'link failed' to 'degraded' and the databases are not replicating.
Running spDiagDRS indicates I have backlogs as incomingmessagesinqueue is 2364, outgoingmessagesinqueue is 107, messages processed in last hour 86, messages sent in last hour 13537 and conflict count 96177.
Prior to running this update all databases were fine.
Looking in rcmctrl.log I cannot see anything obvious other then 3 regular errors which are:
The asynchronous command finished with return message: [A .NET Framework error occurred during execution of user-defined routine or aggregate "spDRSActivation": ~~Microsoft.ConfigurationManager.DataReplicationService.ServiceException: Exception has been thrown by the target of an invocation. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidCastException: Object cannot be cast from DBNull to other types.~~System.InvalidCastException: ~~ at System.DBNull.System.IConvertible.ToInt32(IFormatProvider provider)~~ at Microsoft.ConfigurationManager.DataReplicationService.XmlMessageParser.ExtractChangeData(Hashtable tablesSchema, Int32 sourceSiteNumber, String sourceSiteCode, DrsLogging drsLogging, XmlTextReader changeDataReader, String currentSiteCode, Int32 currentSiteNumber, Boolean needToApplyFilter)~~ at Microsoft.ConfigurationManager.DataReplicationService.XmlMessageParser.ParseSyncDataIntoChangeData(String currentSiteCode, Int32 currentSiteNumber, Int32 sourceSiteNumber, String sourceSiteCode, Boolean needToApplyFilter, Stream messageBody, Hashtable tablesSchema, Int64& lastVersionReceived, Int64& lastSyncVersionToSource, List`1& upsertTableList, Hashtable& upsertData, List`1& deleteTableList, Hashtable& deleteData, Int32& replicationGroupID, DrsLogging drsLogging)~~ at Microsoft.ConfigurationManager.DataReplicationService.MessageHandlerService.ProcessSyncData(Message msgReceived, SqlConnection connection, SqlTransaction transaction, Int32 logLevel, MessageFormat messageFormat, SqlCommand command)~~ at Microsoft.ConfigurationManager.DataReplicationService.MessageHandlerService.ProcessSyncDataXml(Message msgReceived, SqlConnection connect~~ ...~~Microsoft.ConfigurationManager.DataReplicationService.ServiceException: ~~ at Microsoft.ConfigurationManager.DataReplicationService.Service.Run(SqlConnection connection, SqlTransaction transaction, Int32 logLevel, SqlCommand command, Int64 poisonMessageSequenceNumber)~~ at Microsoft.ConfigurationManager.DataReplicationService.MessageHa...].
A few weeks ago I upgraded the SQL from 2012 to 2017 but everything has been fine.
Can anyone point me in the right direction on this please?
Running spDiagDRS indicates I have backlogs as incomingmessagesinqueue is 2364, outgoingmessagesinqueue is 107, messages processed in last hour 86, messages sent in last hour 13537 and conflict count 96177.
Prior to running this update all databases were fine.
Looking in rcmctrl.log I cannot see anything obvious other then 3 regular errors which are:
The asynchronous command finished with return message: [A .NET Framework error occurred during execution of user-defined routine or aggregate "spDRSActivation": ~~Microsoft.ConfigurationManager.DataReplicationService.ServiceException: Exception has been thrown by the target of an invocation. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidCastException: Object cannot be cast from DBNull to other types.~~System.InvalidCastException: ~~ at System.DBNull.System.IConvertible.ToInt32(IFormatProvider provider)~~ at Microsoft.ConfigurationManager.DataReplicationService.XmlMessageParser.ExtractChangeData(Hashtable tablesSchema, Int32 sourceSiteNumber, String sourceSiteCode, DrsLogging drsLogging, XmlTextReader changeDataReader, String currentSiteCode, Int32 currentSiteNumber, Boolean needToApplyFilter)~~ at Microsoft.ConfigurationManager.DataReplicationService.XmlMessageParser.ParseSyncDataIntoChangeData(String currentSiteCode, Int32 currentSiteNumber, Int32 sourceSiteNumber, String sourceSiteCode, Boolean needToApplyFilter, Stream messageBody, Hashtable tablesSchema, Int64& lastVersionReceived, Int64& lastSyncVersionToSource, List`1& upsertTableList, Hashtable& upsertData, List`1& deleteTableList, Hashtable& deleteData, Int32& replicationGroupID, DrsLogging drsLogging)~~ at Microsoft.ConfigurationManager.DataReplicationService.MessageHandlerService.ProcessSyncData(Message msgReceived, SqlConnection connection, SqlTransaction transaction, Int32 logLevel, MessageFormat messageFormat, SqlCommand command)~~ at Microsoft.ConfigurationManager.DataReplicationService.MessageHandlerService.ProcessSyncDataXml(Message msgReceived, SqlConnection connect~~ ...~~Microsoft.ConfigurationManager.DataReplicationService.ServiceException: ~~ at Microsoft.ConfigurationManager.DataReplicationService.Service.Run(SqlConnection connection, SqlTransaction transaction, Int32 logLevel, SqlCommand command, Int64 poisonMessageSequenceNumber)~~ at Microsoft.ConfigurationManager.DataReplicationService.MessageHa...].
A few weeks ago I upgraded the SQL from 2012 to 2017 but everything has been fine.
Can anyone point me in the right direction on this please?
