Due to economical reasons, the traditional philosophy in data centers was to scale out, rather than scaling up. However, the advances in CMP technology enabled chip multiprocessors to become more prevalent and they are expected to become more affordable and power-efficient in the coming years. Current trend towards more densely packaged systems and increasing demand for higher performance push the market towards placing datacenters on highly powerful chips that have many cores on a single platform. However, increasing the number of cores on a single chip brings along very important problems to be addressed at the chip level regarding the use of shared resources and QoS satisfaction. After briefly exploring current datacenter perspective, this paper captures the current state of the art in the field of chip multiprocessors through a detailed discussion of different studies that pave the way to the datacenters on-chip. Finally, a number of open research issues are highlighted with the intention of inspiring new contributions and developments in the field of datacenters on-chip.