Add retry to tests #9016

Job	Run time
test	33m 43s
build	4m 24s
lint	18m 29s
CodeQL	6m 43s
integration-configs-db	2m 22s
integration (requires_docker)	5m 47s
integration (integration_alertmanager)	4m 1s
integration (integration_backward_compatibility)	8m 52s
integration (integration_memberlist)	7m 40s
integration (integration_querier)	12m 14s
integration (integration_ruler)	4m 14s
integration (integration_query_fuzz)	2m 30s
deploy	0s
deploy_website	0s
	1h 50m 59s

Provide feedback