The second piece of Sheet Music is on the rooftops. Looking at the Guardian and the giant, out-of-order elevator, climb the buildings to your right. This is the right side of The Slums.