Paraiyar as a word referring to a occupational group first appears in the second century CE writings of Mangudi Kilar. The Purananuru mentions the Tudiyar, the Panar, the Paraiyar and the Kadambar as one of the four tribes of the Tamil world which should be respected. Robert Caldwell, a nineteenth-century missionary and grammarian who worked in South India, was in agreement with some Indian writers of the same period who considered the name to derive from the Tamil word parai (drum). According to this hypothesis, the Paraiyars were originally a community of drummers who performed at auspicious events like weddings and funerals. As their population increased, they were forced to take up occupations that were considered unclean, such as burial of corpses and scavenging and because of this they came to be considered an untouchable caste. M. Srinivasa Aiyangar, writing a little later, found this etymology unsatisfactory, arguing that beating of drums could not have been an occupation of so many people. Some other writers, such as Gustav Solomon Oppert, have derived the name from the Tamil word poraian, the name of a regional subdivision mentioned by ancient Tamil grammarians, or the Sanskrit pahariya, meaing "hill man".
More recently, George L. Hart's textual analysis of the Sangam literature (c. 300 BCE–300 CE) has led him to favour Caldwell's earlier hypothesis. The literature has references to the Tamil caste system and refers to a number of "low-born" groups variously called Pulaiyar and Kinaiyar. Hart believes that one of the drums called kiṇai in the literature later came to be called paṟai and the people that played the drum were paraiyar (plural of paraiyan).
Hart says that the pulaiyar performed a ritual function by composing and singing songs in the king's favour and beating drums, as well as travelling around villages to announce royal decrees. They were divided into subgroups based on the instruments they played and one of these groups – the Kinaiyan – "was probably the same as the modern Paraiyan". He says that these people were believed to be associated with magical power and kept at a distance, made to live in separate hamlets outside villages. However, their magical power was believed to sustain the king, who had the ability to transform it into auspicious power. Moffatt is less sure of this, saying that we do not know whether the distancing was a consequence of the belief in their magical powers or in Hinduism's ritual pollution as we know of it nowadays.
Burton Stein describes an essentially continuous process of expansion of the nuclear areas of the caste society into forest and upland areas of tribal and warrior people, and their integration into the caste society at the lowest levels. Many of the forest groups were incorporated as Paraiyar either by association with the parai drum or by integration into the low-status labouring groups who were generically called Paraiyar. Thus, it is thought that Paraiyar came to have many subcastes.
During the Bhakti movement (c. 7th–9th centuries CE), the saints – Shaivite Nayanars and the Vaishnavite Alvars – contained one saint each from the untouchable communities. The Nayanar saint Nandanar was born, according to Periya Puranam, in a "threshold of the huts covered with strips of leather", with mango trees from whose branches were hung drums. "In this abode of the people of the lowest caste (kadainar), there arose a man with a feeling of true devotion to the feet of Siva." Nandanar was described as a temple servant and leather worker, who supplied straps for drums and gut-string for stringed instruments used in the Chidambaram temple, but he was himself not allowed to enter the temple. The Paraiyar regard Nandanar as one of their own caste. Paraiyars wear the sacred thread under rituals such as marriage and funeral.
Scholars such as Burchett and Moffatt state that the Bhakti devotationalism did not undermine Brahmin ritual dominance. Instead, it might have strengthened it by warding off challenges from Jainism and Buddhism.
By the early 19th century, the Paraiyars had a degraded status in the Tamil society. Francis Buchanan's report on socio-economic condition of South Indians described them ("Pariar") as inferior caste slaves, who cultivated the lands held by Brahmins. This report largely shaped the perceptions of the British officials about contemporary society. They regarded Pariyars as an outcaste, untouchable community. In the second half of the 19th century, there were frequent descriptions of the Paraiyars in official documents and reformist tracts as being "disinherited sons of the earth". The first reference to the idea may be that written by Francis Whyte Ellis in 1818, where he writes that the Paraiyars "affect to consider themselves as the real proprietors of the soil". In 1894, William Goudie, a Weslyan missionary, said that the Paraiyars were self-evidently the "disinherited children of the soil". English officials such as Ellis believed that the Paraiyars were serfs toiling under a system of bonded labour that resembled the European villeinage. However, scholars such as Burton Stein argue that the agricultural bondage in Tamil society was different from the contemporary British ideas of slavery.
Historians such as David Washbrook have argued that the socio-economic status of the Paraiyars rose greatly in the 18th century during the Company rule in India; Washbrook calls it the "Golden Age of the Pariah". Raj Sekhar Basu disagrees with this narrative, although he agrees that there were "certain important economic developments".
The Church Mission Society converted many Paraiyars to Christianity by the early 19th century. During the British Raj, the missionary schools and colleges admitted Paraiyar students amid opposition from the upper-caste students. In 1893, the colonial government sanctioned an additional stipend for the Paraiyar students. The colonial officials, scholars, and missionaries attempted to rewrite the history of the Paraiyars, characterising them as a community that enjoyed a high status in the past. Edgar Thurston (1855-1935), for example, claimed that their status was nearly equal to that of the Brahmins in the past. H. A. Stuart, in his Census Report of 1891, claimed that Valluvans were a priestly class among the Paraiyars, and served as priests during Pallava reign. Robert Caldwell, J. H. A. Tremenheere and Edward Jewitt Robinson claimed that the ancient poet-philosopher Thiruvalluvar was a Paraiyar.
Iyothee Thass, a Siddha doctor by occupation, belonged to a Paraiyar elite. In 1892, he demanded access for Paraiyars to Hindu temples, but faced resistance from upper-caste Brahmins and Vellalars. This experience led him to believe that it was impossible to emancipate the community within the Hindu fold. In 1893, he also rejected Christianity and Islam as the alternatives to Hinduism, because caste differences had persisted among Indian Christians, while the backwardness of contemporary local Muslims made Islam unappealing.
Thass subsequently attempted a Buddhist reconstruction of the Tamil religious history. He argued that the Paraiyars were originally followers of Buddhism and constituted the original population of India. According to him, the Brahmanical invaders from Persia defeated them and destroyed Buddhism in southern India; as a result, the Paraiyars lost their culture, religion, wealth and status in the society and become destitute. In 1898, Thass and many of his followers converted to Buddhism and founded the Sakya Buddha Society (cākkaiya putta caṅkam) with the influential mediation of Henry Steel Olcott of the Theosophical Society. Olcott subsequently and greatly supported the Tamil Paraiyar Buddhists.
Jean-Antoine Dubois, a French missionary who worked in India between 1792-1823 and had a Brahmin-centric outlook, recorded the community's name as Pariah. He described them as people who lived outside the system of morals prescribed by Hinduism, accepted that outcaste position and were characterised by "drunkenness, shamelessness, brutality, truthlessness, uncleanliness, disgusting food practices, and an absolute lack of personal honour". Moffat says this led to pariah entering the English language as "a synonym for the socially ostracised and the morally depraved".
Iyothee Thass felt that Paraiyar was a slur, and campaigned against its usage. During the 1881 census of India, he requested the government to record the community members under the name Aboriginal Tamils. He later suggested Dravidian as an alternative term, and formed the Dhraavidar Mahajana Sabhai (Dravidian Mahajana Assembly) in 1891. Another Paraiyar leader, Rettamalai Srinivasan, however, advocated using the term Paraiyar with pride. In 1892, he formed the Parayar Mahajana Sabha (Paraiyar Mahajana Assembly), and also started a news publication titled Paraiyan.
Thass continued his campaign against the term, and petitioned the government to discontinue its usage, demanding punishment for those who used the term. He incorrectly claimed that the term Paraiyar was not found in any ancient records (it has been, in fact, found in the 10th-century Chola stone inscriptions from Kolar district). Thass subsequently advocated the term Adi Dravida (Original Dravidians) to describe the community. In 1892, he used the term Adidravida Jana Sabhai to describe an organisation, which was probably Srinivasan's Parayar Mahajana Sabha. In 1895, he established the People’s Assembly of Urdravidians (Adidravida Jana Sabha), which probably split off from Srinivasan's organisation. According to Michael Bergunder, Thass was thus the first person to introduce the concept of Adi Dravida into political discussion.
Another Paraiyar leader, M. C. Rajah — a Madras councillor — made successful efforts for adoption of the term Adi-Dravidar in the government records. In 1914, the Madras Legislative Council passed a resolution that officially censured the usage of the term Paraiyar to refer to a specific community, and recommended Adi Dravidar as an alternative. In the 1920s and 1930s, Periyar E. V. Ramasamy ensured the wider dissemination of the term Adi Dravida.
Paraiyars belong to the Valangai ("Right-hand caste faction"). Some of them assume the title Valangamattan ("people of the right-hand division"). The Valangai comprised castes with an agricultural basis while the Idangai consisted of castes involved in manufacturing. Valangai, which was better organised politically.