optimize GPU for PARsynthizer

[Ng-ms]

I have a 500k raw data and I am using a PARsynthizer to generate data, but I am not able to use all the raw because of the memory the model use and it is not opmitzed for the GPU to be faster
this is the error I got when I am using the GPU
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 37.70 GiB (GPU 0; 15.57 GiB total capacity; 4.60 GiB already allocated; 10.83 GiB free; 4.62 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[npatki]

Thanks for filing the issue @Ng-ms. We will use it to keep track of the problem and provide updates as we make the necessary updates.

We usually prioritize feature requests based on demand and use cases, so if you are able to provide a bit more information about your project, that would be helpful.

In the meantime, I'm noting down a workaround that may be useful for you and for anyone else who comes across the issue.

Workaround: Subsampling Your Training Data

Our previous research has indicated that you may not need to use the full dataset for training an SDV synthesizer -- a subsample of the data may work just as well for creating high quality synthetic data.

For multi-sequence data such as PAR, I recommend that you randomly subsample entire sequences. That is, don't break apart a sequence belonging to an entity.

For example, for the data shown here, you would want to choose random Patient IDs first and then select the entire sequence corresponding to those Patient IDs.

import numpy as np

SAMPLE_SIZE = 100 # number of sequences to sample

# select random Patient IDs first
unique_sequence_keys = data['Patient ID'].unique()
sampled_keys = np.random.choice(unique_sequence_keys, size=SAMPLE_SIZE, replace=False)

# select all rows (full sequence) belonging to those Patients
sampled_data = data[data['Patient ID'].isin(sampled_keys)]