How much GPU energy does fp16 save? Measured on a Tesla T4

How much GPU energy does fp16 actually save over fp32? Spec sheets give TDP and peak FLOPS, not joules per job, so I measured it on a Tesla T4 using the GPU's own NVML energy counter. The job import sys, torch dtype = {"fp16": torch.float16, "fp32": torch.float32}[sys.argv[1]] a = torch.randn(8192, 8192, device="cuda", dtype=dtype) for _ in... Weiterlesen: How much GPU energy does fp16 save? Measured on a Tesla T4