How to profile event distribution in communication server

Procedure Name

How to profile event distribution in communication server.

Procedure Steps

Enable the following flag on the communication server that needs profiling in the acs.ini file in the existing [Service] section and restart communication server

DistributionPerformanceCounter=1

Two log files will be generated, distributionPerf.txt and lnlcomsrvr_eventsink_perf.  

Lnlcomsrvr_eventsink_perf will show a breakdown of the events, the type (visible event or invisible hardware status event) and statistics for each panel they came from.   

This type of profiling will help diagnosing the offending panel(s) in a situation where a large flood of hardware status events occurs in a short amount of time, or when trying to diagnose message bus pipe warnings for too many messages in the publisher queue in LnlLogError.log.

From the LnlLogError.log :

Description: Over 10001 messages in publisher queue. Slowing down new messages retrieval.

This situation can happen during a flood of events to a communication server.

From the extended log lnlcomsrvr_eventsink_perf :

DETAILS:      DEBUG MSG: Event sink: Pipe size: 0, ew:366,

Statuses Stat: [Events: 98% Status: 2% ] |

Panel Stat: [(230: 99.7%)(32767: 0.3%)]

DETAILS:      DEBUG MSG: Event distr: Pipe:5, ew:1486, 20211022 13:59:22, 7.2ms, Last message details: [Panel 230, Device 1, Aux 1, Message Type: EVENT, EventType: 4, EventID: 555, SN: 1635091659]

Pipe / pipe size : How many events exist in internal communication server event queue prior to distributing to log event thread, client event pipe, and message bus pipe.

Ew : How many items arrived from panels since the last report (one minute) and time / rate of distribution.

Panel Stat : PanelID and percentage of events incoming from each panel since last check.

To profile a single visible event the following query can be executed on the database by adding in the serial number and panelID in the where clause.  Note that hardware status events are not visible in the database.

SELECT EVENTS.SERIALNUM,EVENT.EVDESCR as 'Event Description',EVENTS.EVENTID,EVENTS.DEVID as 'Device',EVENTS.MACHINE as 'PanelID',ACCESSPANE.NAME as 'Panel Name',EVENTS.INPUTDEVID as 'INPUT',EVENTS.EVENT_TIME_UTC FROM EVENTS

INNER JOIN EVENT ON EVENTS.EVENTTYPE=EVENT.EVTYPEID AND EVENTS.EVENTID=EVENT.EVID 

INNER JOIN ACCESSPANE ON EVENTS.MACHINE=ACCESSPANE.PANELID

WHERE EVENTS.SERIALNUM = <serialnumber> AND PANELID = <panelid>

distributionPerf.txt profiling is useful when there is a suspected delay delivering events to thick clients.  The log will show the serialnumber of the event in question and the time to deliver the event once received from the hardware.

[Lnlcommsrvr] SendMessageToAllClients 100 sn:1635075543 took 16 ms

Applies To

OnGuard 7.6 Update 2 and later

Additional Information

© 2024 Honeywell International Inc. All Rights Reserved.