q abt query of data about servers and clients

[iFriendGit]

First of all, thank you so much for Jamulus! I can't say enough how much joy and fun it has given me!

Second, I'd like to know if it's ok to query the genre servers (e.g. https://explorer.jamulus.io/servers.php?central=anygenre1.jamulus.io:22124) for the JSON data about servers and clients for further processing in a script/program. Or is there any other, more efficient way to get the server data?
If it's ok to query, then I'd like to know what the limits are, e.g. how about querying every genre once a second (just as a very extreme example)? And what if a lot of people would do this simultaneously?

Thank you in advance for you kind answer.

[hoffie]

@softins is running Jamulus Explorer, so he'd be most qualified to answer this (I guess he may see this ping and reply here).
Another option would be running the code base yourself -- it is open source:

https://github.com/softins/jamulus-web

You should still be careful about not overloading the scraped servers.

[softins]

Yes, it's ok to query the backend for JSON data. That is what https://jamulus.live does (not my project).

It's also ok to run your own copy of jamulus-php.

To avoid overloading directories and servers, the backend caches the JSON for each directory (genre) for about 10 seconds. If the cached data is less than 10 seconds old, it is returned directly without querying the directory and all listed servers. You will know that has happened, because the answer comes back immediately. If it needs to query the directory and servers, the answer will take a few seconds to come back.

[iFriendGit]

Thank you very much, guys!

FYI: the background of my q is that I have a script and program that periodically queries the genre servers one by one for further processing. Now I'm thinking about making it available, but am worried it could overload your fine systems if used on a bigger scale.

That was the reason for the other part of my q: "If it's ok to query, then I'd like to know what the limits are, e.g. how about querying every genre once a second (just as a very extreme example)? And what if a lot of people would do this simultaneously?"

Any thoughts on that? Would you suggest I do smth to avoid overloading? Send a flag so server could ignore, limit the period, whatever...?

I'm very much a hobby programmer, so pls excuse any ignorant stuff.

[pljones]

The first question you need to ask and answer is "What am I trying to achieve for the user of the program?". That's not necessarily what you're asking your question about here - so you need to check you know that first :).

The next point of reference is the existing code base - that's the client connect dialogue in this case, which refreshes one server list at a time only and does that infrequently enough that a user can read the list content comfortably. Again: think about how the user will use the program: if you refresh too often, it won't be pleasant for users to use.

The timeout for the server list, per directory is SERV_LIST_REQ_UPDATE_TIME_MS -- 2000ms. That gives someone two seconds to read up to 150 entries, which isn't very long..! So you might want to think about what you're trying to achieve. If you're going to update for every directory once a second then it's going to be totally unusably, I'd say, so pointless.

Take a look at jamulus.live -- the update time there is pretty long.

You also have to allow for the fact that the response to the UDP request won't be instant - and will be longer depending how far you are from the directories.

As Tony has said, caching will help you. If you have many users of your site, that doesn't mean you request the lists each time any one of those users request the list from you. It means you request the lists when your local cache goes stale and serve that cache to all your users. You can massively improve performance that way.

[iFriendGit]

Thank you very much @pljones, for your kind information!
Now I see that I should have stated from the beginning what the thing about my script/program is all about. So I will do it now and apologize for omitting it before.
I call my thing the "Jamulus Buddies Tracker". Summary: you have a list of buddies (text file) and it will show you on a very small on top window where they are, avoiding the need for switching different genres with the J client and scanning visually.
My question about querying ever second and so was more theoretical, to see what happens if A LOT of queries are received at the directory servers at once. I have my program running with a period of 50s and that is absolutely enough.

Dev versions of course!

[pljones]

That's pretty similar to what jamulus.live already offers, to be honest. You can set up people you want to track so they're highlighted if they're on, and it shows which server they're on.

[iFriendGit]

I know, and still also quite different in some ways, I think: local, no reg, small window thus very small real estate on screen, very succinct info about buddies. That factors were very important to me at least.
Anyway, it's just a hobby project that I was thinking about sharing. Thank you very much for your feedback!

[pljones]

Ah right. Still, I'd not update too often - if you know someone is around, you can often find them just by typing their name into the filter. No need to update frequently to keep track of exactly where they are across all directories. So yes, go ahead read all the directories to start with but then you probably don't need to update more than once a minute, maybe even less often.

[iFriendGit]

Thank you, I understand, and I'm thinking about just leaving the period hardcoded to a reasonable minimum.

The motivation for doing this app was exactly to avoid all that stuff just to know if someone is around:

1. open J
2. open the connection setup
3. go through all genres with the J client (filter or not)
4. repeat every x time
   Sometimes repeat the above for every buddy (I have quite the long list).
   Alternatively there is the online site, but as already said it has other downsides and not the same functionality.

My app:

1. one click and enjoy :)
   And it even keeps a log

So for me personally at least there is a huge benefit to it :)