Replace pystan models with numpyro

[jack89roberts]

See:

• Experiment with using Pyro instead of Stan? #313
• Integration with AIrsenal anguswilliams91/bpl-next#4

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[jack89roberts]

Hopefully have a working team model now, but could do with someone checking what I've done to avoid NaNs and for adding new teams.

Also have an idea to get around the multiprocessing issue - we should be able to pre-generate score probabilities for each fixture and pass those to each thread rather than the team model (may also be a bit quicker as we'd only compute them once per fixture rather than once per player like we do currently)

[jack89roberts]

Nice work Nick!

Quick comparison before/after new player model (but using bpl-next in both cases):

Stan player model:

==================================================
PREDICTED TOP 5 PLAYERS FOR GAMEWEEK(S) [1, 2, 3, 4, 5]:
==================================================
GK:
1. Alisson Ramses Becker, 21.38pts (£6.0m, LIV)
2. Emiliano Martínez, 21.24pts (£5.5m, AVL)
3. Dean Henderson, 20.28pts (£5.0m, MUN)
4. Illan Meslier, 19.69pts (£5.0m, LEE)
5. Hugo Lloris, 19.56pts (£5.5m, TOT)
-------------------------
DEF:
1. Trent Alexander-Arnold, 28.34pts (£7.5m, LIV)
2. Andrew Robertson, 25.72pts (£7.0m, LIV)
3. Lucas Digne, 20.69pts (£5.5m, EVE)
4. Nathaniel Phillips, 20.48pts (£4.5m, LIV)
5. Rúben Santos Gato Alves Dias, 20.11pts (£6.0m, MCI)
-------------------------
MID:
1. Mohamed Salah, 31.83pts (£12.5m, LIV)
2. Sadio Mané, 28.63pts (£12.0m, LIV)
3. Bruno Miguel Borges Fernandes, 27.17pts (£12.0m, MUN)
4. Heung-Min Son, 25.54pts (£10.0m, TOT)
5. Mason Greenwood, 21.23pts (£7.5m, MUN)
-------------------------
FWD:
1. Harry Kane, 27.45pts (£12.5m, TOT)
2. Jamie Vardy, 27.02pts (£10.5m, LEI)
3. Gabriel Fernando de Jesus, 26.97pts (£8.5m, MCI)
4. Kelechi Iheanacho, 25.02pts (£7.5m, LEI)
5. Roberto Firmino, 23.17pts (£9.0m, LIV)
-------------------------

NumPyro player model:

==================================================
PREDICTED TOP 5 PLAYERS FOR GAMEWEEK(S) [1, 2, 3, 4, 5]:
==================================================
GK:
1. Alisson Ramses Becker, 21.38pts (£6.0m, LIV)
2. Emiliano Martínez, 21.24pts (£5.5m, AVL)
3. Dean Henderson, 20.28pts (£5.0m, MUN)
4. Illan Meslier, 19.69pts (£5.0m, LEE)
5. Hugo Lloris, 19.56pts (£5.5m, TOT)
-------------------------
DEF:
1. Trent Alexander-Arnold, 29.34pts (£7.5m, LIV)
2. Andrew Robertson, 26.32pts (£7.0m, LIV)
3. Nathaniel Phillips, 21.18pts (£4.5m, LIV)
4. Lucas Digne, 21.04pts (£5.5m, EVE)
5. Rúben Santos Gato Alves Dias, 20.64pts (£6.0m, MCI)
-------------------------
MID:
1. Mohamed Salah, 33.57pts (£12.5m, LIV)
2. Sadio Mané, 30.80pts (£12.0m, LIV)
3. Bruno Miguel Borges Fernandes, 28.37pts (£12.0m, MUN)
4. Heung-Min Son, 27.06pts (£10.0m, TOT)
5. Mason Greenwood, 23.01pts (£7.5m, MUN)
-------------------------
FWD:
1. Gabriel Fernando de Jesus, 29.71pts (£8.5m, MCI)
2. Harry Kane, 28.02pts (£12.5m, TOT)
3. Jamie Vardy, 27.98pts (£10.5m, LEI)
4. Roberto Firmino, 26.30pts (£9.0m, LIV)
5. Kelechi Iheanacho, 26.26pts (£7.5m, LEI)
-------------------------

New model seems to like Jesus and Firmino a decent chunk more than the old model (NumPyro scores also look higher in general).

[nbarlowATI]

Interesting!!
I'm going to tidy up a bit (flake8 etc.) and add a new function to the model to return the p(score), p(assist) etc. given a player_id. I'm still not entirely sure what it's doing for new players where it doesn't have any previous data...

[jack89roberts]

I think we just predict 0 for new players as they have 0 recent minutes, so they probably don't make it as far as the player model.

The model seems to predict Firmino and Salah to have very similar score/assist probabilities - I'm wondering if it's something to do with priors (we have a position-dependent prior and Salah scores much more than an average midfielder but Firmino (maybe) much less than an average striker - perhaps the prior is dragging them both towards the mean).

[nbarlowATI]

Hmm... that would make sense, but not sure why it would be any different to what it was with the Stan model...?

For the new players, I think you're right - it's the minutes that gives them zero. I looked at what the model itself predicts for their p(score) etc. though, and it's very close to the position-dependent averages, which I guess is what we want.