Cusco:
Loki is a hostel chain where you can share your room with a group of people for as low a $8 US per bed. They also have private rooms from $27 US per room
Home Sweet Home This hostel is in both Cusco and Arequipa. The prices are quite cheap according to their website.
Casa Andina is the Hotel chain that Garden House recommended in Cusco