Developing a Tagset for Machine Learning based POS Tagging in Punjabi

The research in European and East Asian languages for part of speech (POS) tagset design has evolved from simple listing of morphosyntactic features in one language to standard tagset (Penn tagset) and common framework for multiple languages like Expert Advisory Group on Language Engineering Standards (EAGLES) guidelines etc. A number of tagsets and tagged corpus have been developed for these languages for furthering research. During our study, we found that most of the research done on tagset design is for English. But for Indian languages (IL), very little work has been done on tagset design. For Punjabi, only one tagset was developed, which consists of 630 fine-grained tags. We developed a POS tagset by using coarse-grained granularity for representing morphosyntactic features of Punjabi and devised the various tags for the proposed tagset. The proposed tagset is then compared with the existing tagsets of the ILs. As compared to IL tagsets, particularly Punjabi, our tagset has 38 tags. The proposed tagset will be used for the development of machine learning based POS tagger for Punjabi.