The caves in the seaside cliffs of Changbin Township were formed millions of years ago by sea erosion. Originally underwater, they were puched 150 meters above sea level by tectonic movement. There are over a dozen separate caves on the cliffs, all of which are connected by walkways. The largest one is Lingyen Cave, and the highest is Kunlun Cave. The other caves are named Cyanyuan, Jhaochen, Hailei, Chaoyin, Yongan, and Shueilian. Altars have been erected in most of the caves for people to worship. Prehistoric artifacts have been discovered here, making the cave one of the oldest prehistoric sites in Taiwan and earning them the designation as a national historic site of the first rank.