ChannelFinder returns variable count of PVs when IOCs are connected/disconnected

[connorfi]

I am writing a utility that polls ChannelFinder for changes via the RESTful API. I am seeing that the total returned entries for a given query varies for a time after an IOC (with RecCaster) has connected or disconnected. I am running version ChannelFinder 4.7.1 w/ elasticsearch 8.7.1. Here is the output from my polling, latest to oldest responses.

Updated PV Lists - 03:16:33 PM SPVs: [1657], ConfigPVs: [593], CPVs: [398]
Updated PV Lists - 03:16:21 PM SPVs: [5], ConfigPVs: [39], CPVs: [0]
Updated PV Lists - 03:16:20 PM SPVs: [5], ConfigPVs: [53], CPVs: [0]
Updated PV Lists - 03:16:19 PM SPVs: [74], ConfigPVs: [97], CPVs: [2]
Updated PV Lists - 03:16:18 PM SPVs: [151], ConfigPVs: [121], CPVs: [20]
Updated PV Lists - 03:16:17 PM SPVs: [237], ConfigPVs: [147], CPVs: [42]
Updated PV Lists - 03:16:15 PM SPVs: [322], ConfigPVs: [174], CPVs: [65]
Updated PV Lists - 03:16:14 PM SPVs: [414], ConfigPVs: [202], CPVs: [86]
Updated PV Lists - 03:16:13 PM SPVs: [523], ConfigPVs: [235], CPVs: [114]
Updated PV Lists - 03:16:11 PM SPVs: [631], ConfigPVs: [269], CPVs: [139]
Updated PV Lists - 03:16:10 PM SPVs: [739], ConfigPVs: [301], CPVs: [164]
Updated PV Lists - 03:16:08 PM SPVs: [851], ConfigPVs: [337], CPVs: [194]
Updated PV Lists - 03:16:06 PM SPVs: [956], ConfigPVs: [369], CPVs: [221]
Updated PV Lists - 03:16:04 PM SPVs: [1081], ConfigPVs: [408], CPVs: [251]
Updated PV Lists - 03:16:02 PM SPVs: [1212], ConfigPVs: [449], CPVs: [283]
Updated PV Lists - 03:16:00 PM SPVs: [1351], ConfigPVs: [491], CPVs: [318]
Updated PV Lists - 03:15:58 PM SPVs: [1495], ConfigPVs: [535], CPVs: [353]
Updated PV Lists - 03:15:26 PM SPVs: [1657], ConfigPVs: [593], CPVs: [398]

I have about 2000 PVs in Channelfinder. The IOC that I am start/stopping hosts all of the PVs. I assume this "re-indexing" would occur anytime an IOC w/ RecCaster connected to CF is started or stopped. The re-indexing takes about 30 seconds to return to the original count.

Is this behavior known/expected? Is it resolved/improved in 4.7.2. Issue #96 seems to discuss some changes that might be related.

[jacomago]

Hi @connorfi

I recommend you use 4.7.2. We at ESS went from syncing all 1500 IOCs (around 2 million PVs I think) over several days to syncing in less than an hour.

[connorfi]

@jacomago thank you for responding. I have a few more questions to understand the syncing behavior.

1. Under what conditions/when does a sync occur?
2. What is the recommended behavior for processes that rely on ChannelFinder during a sync?
3. Is there a way to detect a sync in progress other than variable responses? For example a "Sync Status" field in GET http://localhost/ChannelFinder to indicate syncing progress/completion?

[jacomago]

1. This is actually controlled on the recceiver side:

• IOC goes up

• recsync module communicates with recceiver

• Note there is a configurable TIMEOUT field for this communication on the IOC

• recceiver tells Channelfinder via REST api data has changed

• ChannelFinder updates elastic with new info

• IOC goes down

• recceiver sees a timeout in communication with IOC

• recceiver tells Channelfinder via REST api data has changed

• ChannelFinder updates elastic with new info

ChannelFinder itself is quite "dumb" by design. It has no idea about what an IOC is, just a channel, property, and tag.

2. With the latest version, I can't expect you to see the middle of a sync. For a single IOC of < 10000 PVs it takes less than a second.

3. In some ways you can always assume ChannelFinder data is in flux? Unless you expect 100% uptime of every IOC and network card, etc.

[connorfi]

Thank you for the additional background.

The use case of a single IOC w/ ~2000 PVs is just for my dev environment. The application will have many more IOCs and PVs, so I would like to understand more about the concept of operations when an installation is waiting for CF to sync. Is it simply a known down-time for services/systems that rely on CF?

I guess the hang-up I have is that I didn't expect CF to tell me that it knows about fewer PVs than it did previously if the channels were not intentionally removed from the DB. In my mind, an IOC coming online/offline would be more of a "DB entry update" (of say the pvStatus property) rather than a delete/add operation. Can you provide some insight into why Channel Finder has this behavior in the first place?