Problems with unsupervised classification GPU load

[patrickmonari]

Hi! I'm trying to run the autoencoder + call features unsupervised classifier, but it keeps aborting because of insufficient memory on my GPU (RTX 3060). Is there a way to run this classifier without buying a better graphics card, or will I end up just having to use the call features only, instead of the autoencoder?

Thanks!
Patrick

[DrCoffey]

Hey Patrick, can you give me a few more details? Does it crash during VAE training? How big is the training set? Does it work with half the data?

I am using a 2060 for most things and haven't run into any memory issues yet.

[patrickmonari]

Hey!

It seems to be crashing during training - it can load the files but won't proceed after that. It works on a smaller set (the one I'm training on is probably ~5,000 calls). I have no idea what I'm talking about, but is there a way to implement parallel processing for this kind of thing?

Cheers,
Patrick

[patrickmonari]

Figured out the issue! Training automatically includes rejected calls, and there were a lot in this set. Pre-removing them reduced the size of the training set by an order of magnitude lol.

It might be nice to have a way to batch-remove rejected calls from detection files. Unless I'm missing something, it seems like the only way to do it right now is by opening each file one-by-one.

Cheers,
Patrick

[DrCoffey]

Hey, there is a tool in the tools menu for removing rejected calls. Clustering should do that automatically though; I may have dropped that when integrating the two versions. Rejected calls really complicate clustering.

…

________________________________ From: Patrick Monari ***@***.***> Sent: Sunday, May 14, 2023 11:55:41 AM To: DrCoffey/DeepSqueak ***@***.***> Cc: mrcoffey ***@***.***>; Comment ***@***.***> Subject: Re: [DrCoffey/DeepSqueak] Problems with unsupervised classification GPU load (Discussion #198) Figured out the issue! Training automatically includes rejected calls, and there were a lot in this set. Pre-removing them reduced the size of the training set by an order of magnitude lol. It might be nice to have a way to batch-remove rejected calls from detection files. Unless I'm missing something, it seems like the only way to do it right now is by opening each file one-by-one. Cheers, Patrick — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/DrCoffey/DeepSqueak/discussions/198*discussioncomment-5898433__;Iw!!K-Hz7m0Vt54!jDGm_TrmpanxYikSdd81Yp70GMd4qSTtmM349XACME9c2g_vLiWTlmlUvR79wzfgUKBwEjpCJA0l99lvEUg_zTtq$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AJOFGE5F6HGQMVTGJDMHJN3XGETC3ANCNFSM6AAAAAAX3WSC4M__;!!K-Hz7m0Vt54!jDGm_TrmpanxYikSdd81Yp70GMd4qSTtmM349XACME9c2g_vLiWTlmlUvR79wzfgUKBwEjpCJA0l99lvEbBQ4TOR$>. You are receiving this because you commented.Message ID: ***@***.***>