Efforts to understand human vision have largely focused on our abilities to perceive the geometric properties of objects such as shapes, sizes, and distances, and have neglected the perception of materials. However correctly perceiving materials is at least as important as perceiving objects, and human vision allows us to tell if objects are hard or soft, smooth or rough, clean or dirty, fresh or spoiled, and dead or alive. Understanding the perception of material properties is therefore of critical importance in many fields. In this talk I will first show how we have been using image synthesis techniques to develop psychophysical models of material perception that can relate the physical properties of materials to their visual appearances. I will then describe how we have been taking advantage of the limits of material perception to develop new techniques for efficiently rendering complex scenes. Finally I will discuss some recent efforts to develop advanced display systems that allow more realistic visualization of complex objects and materials, and allow hands-on interaction