A friend asked me if I had used a GPU for my tests described in blog post “Quickpost: The Electric Energy Consumption Of LLMs”. Because he had tried running an LLM on a machine without GPU, and it was too slow.
I did a quick test, just redoing previous test but without GPU (by setting environment variable CUDA_VISIBLE_DEVICES=-1).
Answering my query took 17 seconds, and required 1.13 Wh (again, for the whole PC).
Quickpost info
Didier Stevens 
What device are you using to measure wattage for the LLM, etc?
It would be nice if it’s a cmd line but if it’s a device; where would someone get that?
Thank you for the post; great info!
Comment by Anonymous — Saturday 12 October 2024 @ 15:05
You can’t do this from the command line, I use a specialized powermeter: a GW Instek GPM-8310.
Comment by Didier Stevens — Tuesday 15 October 2024 @ 16:30