Does your GPU util peak at 100%? If not, increase the batch size until it does (roughly). A couple ideas: do any transforms on the GPU or before starting your training job, have the CPU be solely responsible for loading images from disc. Increase the number of workers to the number of CPUs you have.
Does your GPU util peak at 100%? If not, increase the batch size until it does (roughly). A couple ideas: do any transforms on the GPU or before starting your training job, have the CPU be solely responsible for loading images from disc. Increase the number of workers to the number of CPUs you have.