Fetching addPacket for removed entity after using portal gun to return from the end

[boxfriend]

When using the portal gun to return to the overworld from the end, after loading back into the overworld, the server console starts spamming "[Server thread/WARN] [net.minecraft.entity.EntityTrackerEntry]: Fetching addPacket for removed entity" and the server starts lagging horribly

I'm using the Direwolf20 for 1.12 pack, latest version which uses Minecraft version 1.12.2, forge-12.23.4.2760 and MeeCreeps-1.12-1.2.2

This seems to only happen when coming back from the end, going to the end did not produce this issue, neither did going to or from the twilight forest. Restarting the server appears to be the only way to stop the lag and console spam. I have tried keeping the end chunk loaded thinking that perhaps the chunk unloads and the entity is still loaded in memory, but that made no difference.

Here is my server log after loading the server, taking a portal to the end, and then back
latest.log

I do also get this message in the client-side log which may be relevant "[main/INFO]: Unloaded world <net.minecraft.client.multiplayer.WorldClient@63527b21 0 MultiplayerChunkCache: 441, 441> is still hanging around after 60 seconds."