It stands on a round hillock and was built in 1499 by the 14th Drukpa hierarch, NgawangChoegyel, after the site was blessed by the "Divine Madman" the maverick saint Drukpa Kunley (1455–1529) who built a chorten on the site.
In founding the site it is said that Lama Kunley subdued a demon of Dochu La with his “magic thunderbolt of wisdom” and trapped it in a rock at the location close to where the chorten now stands. He was known as the "Mad Saint" or “Divine Madman” for his unorthodox ways of teaching Buddhism by singing, humour and outrageous behaviour, which amounted to being bizarre, shocking and with sexual overtones. He is also the saint who advocated the use of phallus symbols as paintings on walls and as flying carved wooden phalluses on house tops at four corners of the eaves. The monastery is the repository of the original wooden symbol of phallus that Kunley brought from Tibet. This wooden phallus is decorated with a silver handle and is used to bless people who visit the monastery on pilgrimage, particularly women seeking blessings to beget children.

Behind the temple sits the black chorten constructed on the dog’s burial spot by the Lama to prevent the evil spirit from rising again. This is the oldest recorded historical structure in Chimi Lhakhang village.
Strategically located around the outside of the village are a number of other chortens, which form a type of circle around the central chorten. A common characteristic of the chortens found in this area is that they are all smaller versions of a Khangzangchorten, meaning they are built in a cubical shape. These chortens were mostly built to suppress and control evil spirits or to avert the harm of the spirits away from the community.

A small cubical chorten amid the white prayer flags located a few minutes walk towards the Punatsangchhu river along the farm road south of the Pangna hamlet is known as the Pangkhachorten. A local elder, Ghalley, says that an old man from the locality built the stupa, with Lama Drukpa Kunley’s traditional bamboo plate (bangchung) inside as a relic.
There are a number of historical sites regarded as nyeor sacred sites in Chimi Lhakhang village. These sites are usually natural elements, such as water bodies, rocks or trees that have a special linkage with a spirit or a legend.
For instance, there are several stones along the Punatsangchhu river and around the village that are associated with Drukpa Kunley and regarded as sacred19.

Located to the south-east of the Chimi Lhakhang temple, below the Pangna hamlet, is a pond believed to represent the drupchuor holy water spring of Lama Drukpa Kunley. This is said to be the most sacred water in the Chimi Lhakhang village. According to LopenKunzangThinley’s work on the biography of Drukpa Kunley, the Lama extracted the water from the ground using his miraculous power. This holy spring water is believed to have originally emerged somewhere behind the main temple and to have moved to its current location in order to avoid risk of contamination from impurities. In the past, a donkey would carry water from the pond for offerings in the temple. However, since the donkey’s death, the practice of fetching this water for offerings has been discontinued.
A similar drupchu, also associated with Lama Drukpa Kunley, is the Wolakhadrupchu located in the north of Yuwakha, next to the highway at the base of the hill on which Wolakha village stands.