Experimental Physics and
Industrial Control System

I traced it down to ca_add_masked_array_event/ca_create_subscription as well.

In a recent run, I had 14 out of 2600 sequences not getting their monitor callbacks. I did a dbel on the non-functioning PV’s (H75 H73 H82 H78 H72 L52 L54 P93 R71 R80 V82 V71 V86 Z80)  and a couple of functioning ones (A01 A75). The results are pasted below. The ones that are not getting their monitor callbacks called seem to have a fair number of “undelivered” or “discarded by replacement” messages. The ones that are functioning have “queue empty”.  Is this expected?

Any suggestions as to next steps for debugging are appreciated.

Regards,

Murali

epics> < /tmp/dbel.txt

dbel H75  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=48, thread=0x7f1058000b60, unused entries=32, discarded by replacement=28, duplicate count =94

, ev 0x7f1094013890, ev que 0x7f10a4008520, ev user 0x7f105400ed70

dbel H73  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=48, thread=0x7f1058000b60, unused entries=32, discarded by replacement=28, duplicate count =94

, ev 0x7f1094013980, ev que 0x7f10a40074f8, ev user 0x7f105400ed70

dbel H82  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=48, thread=0x7f1058000b60, unused entries=32, discarded by replacement=28, duplicate count =94

, ev 0x7f10940138e0, ev que 0x7f10a4008520, ev user 0x7f105400ed70

dbel H78  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=48, thread=0x7f1058000b60, unused entries=32, discarded by replacement=28, duplicate count =94

, ev 0x7f1094013840, ev que 0x7f10a40074f8, ev user 0x7f105400ed70

dbel H72  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=76, thread=0x7f1058000b60, unused entries=52, duplicate count =75

, ev 0x7f10940139d0, ev que 0x7f10a4009548, ev user 0x7f105400ed70

dbel L52  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=76, thread=0x7f1058000b60, unused entries=52, duplicate count =75

, ev 0x1ed5230, ev que 0x7f109c009670, ev user 0x7f105400ed70

dbel L54  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=76, thread=0x7f1058000b60, unused entries=52, duplicate count =75

, ev 0x1ed5280, ev que 0x7f109c00a698, ev user 0x7f105400ed70

dbel P93  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=76, thread=0x7f1058000b60, unused entries=52, duplicate count =75

, ev 0x7f1090014670, ev que 0x7f10700137c8, ev user 0x7f105400ed70

dbel R71  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=76, thread=0x7f1058000b60, unused entries=52, duplicate count =75

, ev 0x7f1090010ac0, ev que 0x7f106c01f178, ev user 0x7f105400ed70

dbel R80  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=49, thread=0x7f1058000b60, unused entries=32, discarded by replacement=27, duplicate count =94

, ev 0x7f1090010a20, ev que 0x7f106c0211c8, ev user 0x7f105400ed70

dbel V82  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=32, thread=0x7f1058000b60, unused entries=32, discarded by replacement=44, duplicate count =93

, ev 0x7f10a0024e10, ev que 0x7f105000bd00, ev user 0x7f105400ed70

dbel V71  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=32, thread=0x7f1058000b60, unused entries=32, discarded by replacement=44, duplicate count =93

, ev 0x7f10a0024dc0, ev que 0x7f105000bd00, ev user 0x7f105400ed70

dbel V86  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=32, thread=0x7f1058000b60, unused entries=32, discarded by replacement=44, duplicate count =93

, ev 0x7f10a0024e60, ev que 0x7f105000bd00, ev user 0x7f105400ed70

dbel Z80  10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM } undelivered=76, thread=0x7f1058000b60, unused entries=52, duplicate count =75

, ev 0x7f109c038ee0, ev que 0x7f1094028f80, ev user 0x7f105400ed70

#### Functioning PV’s

dbel A01 10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM }, thread=0x7f1058000b60, queue empty, ev 0x7f105800b4e0, ev que 0x7f105400ed70, ev user 0x7f105400ed70

dbel A75 10

1 PV Event Subscriptions ( monitors ).

VAL { VALUE ALARM }, thread=0x7f1058000b60, queue empty, ev 0x7f1058009e10, ev que 0x7f1070006dc0, ev user 0x7f105400ed70

epics>