In many cases, the primary power consumer on an Arduino is the power LED. Remove it.
Buck converters are more efficient at high loads, but sometimes have higher standing/quiescent losses. You'll need to select carefully.
At reduced clock speeds, most ATMega parts can operate from 2.7 to 5.5V. Depending on what other parts you have, you may be able to operate directly from a 1S lithium cell with no converter or LDO. You'll still need a battery protection chip/PCB to prevent over discharge.
The other thing you want to do is put the micro to sleep for as long as possible.