Can the other designated reviewers (department chairs or deans) see the letters from the other units?

Yes. Access to documents is controlled by the role one holds. Thus if someone is a department chair, they will see all of the departmental recommendations and departmental ad hoc reports.