Routine Jam Forecaster

[mcfnord]

On the website Jamulus.Live, I've added a "coming up" section that only appears occasionally, right after the server boxes. The goal is to provide tips that could improve user experiences. This code tries to detect groups that gather weekly at the same time, perhaps on the same server. The feature uses historical data that I cook regularly. I'd like to re-write this code in a way that is cleaner and more maintainable, but I'm not sure what that might look like. I'm not even quite sure how to optimally design such an algorithm.

The raw data I use is available at these URLs:

• Census Metadata: Information about musicians
• Census Uniq: Timestamped participant details for every jam on every public server, very long file
• Servers Metadata: Information about servers

(I define unique account as being a name-instrument-country combination. A guid is a hash of the name, instrument, and country. The click-name feature in the UI just uses name and country, but adding instrument increases the likelihood that I'm tracking the same person. Our absence of centralized identity management is a strength but has forced me to handle identity as a hash of this pretty-unique detail combination.)

I waited a long time before logging any kind of historical data, but I decided I could create some useful features if I logged this much. The cooked data product updates as JSON here every 5 minutes. It might be empty when you look. It's a JSON description of predicted gathering(s).

I've been using this feature for a few weeks and it has both reminded me of groups I know about and try not to miss, and helped me discover others. It just shows the core people who show up consistently (plus instrument), and will mention a server name if it is consistent, which is usually enough information. Sometimes I want to mention a city, but maybe you just know your friends, and you know which instruments are interesting, and city help isn't so important.

I don't have a title or help for this feature. When you first see it, maybe it's strange, but if you watch it over time, you'll see the times at the left going down, and eventually--hopefully--signal arrival of that group at that time.

I estimate that the predictions are reasonable about 80% of the time, but of course sometimes people break patterns they've been in. Don't think I can fix that. But maybe I introduce a little tension if you're apparently expected for a jam you didn't schedule or mention to anyone but your little crew. Commercial competitors have manual jam scheduling systems and I thought automated inferences would be a lower-maintenance equivalent.

This v1 shows the same stuff for everyone. But there are ways to determine a user's location and maybe even Jamulus identity, so a custom data product isn't out of the question.

I guess I could show a "History" section that shows where you or your friends spent time lately, but I doubt that's useful information. I avoid "infoporn" that looks interesting but probably won't improve outcomes. History seems like history. Why bother? Geography seems a little more valuable as a proxy for latency.

Maybe you have some ideas about patterns I can reveal that improve outcomes.

And maybe you know a more appropriate framework for processing and querying this kind of data.

I'm also tracking per-day use now in the FAQ section, and can report we've reached 100 unique browsers daily that stay at least 2 minutes. That milestone means a lot because I remember when I had 1 unique browser (Veronica the Canadian nurse, my first Jamulus friend and first user!). Also thankful to Paulo for helping me translate key server details into Italian! I'm a huge fan of so many musicians here and benefit from all the support. A year ago I quit Google because they were indifferent to my wins while this community was cheering me on. My goal is to provide tools that fuel healthy growth in our community. I hope now there's one more passive tool that helps more people find the best jams as both participants and listeners.

[ann0see]

Hi @mcfnord is this clearly labelled that it's just a prediction based on old data? Probably that's the first step towards clearing confusion.

[mcfnord]

Could add a label. But in my Twilight Zone mind, I imagine the predictions driving people's activities just like people's activities drive predictions! :)

[chrisrimple]

Perhaps add a feature allowing recurring jam participants to "verify" your prediction of a jam's regular date/time/server? Or to mark it as incorrect? Then filter appropriately in your results, with a "confirmed" section at the top? For truly "open jams", you might also publicize them to non-musicians for "listening times", if you're monitoring those jams with your streaming bot.

[mcfnord]

When predicted groups gather, I could add an "as foretold" sash.

[mcfnord]

I think I can add a History tab that shows your jam history. It could be reverse-chronological servers quite similar to those in the live list, but including time and duration of your visit. I think this is useful because sometimes I wonder, "Who was that I jammed with this morning?" Could get long because everyone who joined you there would be listed.

[mcfnord]

So... I also have the data necessary to test if the predictions came true. Which means I can test competing algorithms. So I kind of need a more proper data architecture and a machine learning harness, but just testing predictions can guide the rest.

I bet I could double the predictive accuracy this way. And it kind of seems worth it, because I think the information does lead to better outcomes. I'm using it successfully. I was with a group tonight that saw they had been predicted. They enjoyed being "announced" in this fashion.

[pljones]

explorer.jamulus.io is no more official than any other site reading server lists from directories. I don't see anything being said that Jamulus, itself, would be affected.

If you do not want your server information visible to any client that can read server lists from a directory, do no register with that directory. The public directories are public and server information for servers registered with them is visible to any such client.

[rdica]

I misunderstood the status of explorer.jamulus.io. Site documentation directly references explorer.jamulus.io in Server Troubleshooting, its natural to conclude this site is official.

[pljones]

It's a useful tool for diagnosing problems, hence it's mentioned. Any third party tool that's useful could be mentioned.

[mcfnord]

You can run servers.php on a simple Linux node, and ask Tony to include your data as an option in Jamulus Explorer. It just runs nginx with PHP, which I can help you set up. Then you could also ask Tony to remove the San Francisco server from his options, and use your source instead. Previously I tried using multiple servers.php datasources because there are occasional blankout conditions, but I determined these blankouts occur across all servers.php instances at the same time. (Still a mystery worth solving!)

[ann0see]

@rdica just to clarify: softins hosts explorer.jamulus.io as his project and is maintainer of the Jamulus Software.
mcfnord is contributor to the Jamulus Software but not maintainer. As personal project he hosts Jamulus.live