If the mosque was built because of those graves, then it is not acceptable to pray in it; this is like what the Jews and Christians did who used to venerate graves and build places of worship over them.
If that is not the case and the mosque was not built because of those graves, then it is acceptable to pray in it, but some scholars stipulate that between the wall of the mosque and the graves there should be some empty space or a distance separating them from one another.
Shaykh al-Islam Ibn Taymiyah (may Allah have mercy on him) said:
Al-Aamidi and others said: it is not permissible to pray in it - i.e., a mosque whose qiblah faces towards graves - unless there is between the wall and the graveyard another barrier. And some of them mentioned that this was stated by Ahmad.
Al-Mustadrak ala Majmoo al-Fataawa, 3/75
In al-Durar al-Saniyyah fil Ajwabah al-Najdiyyah (4/265) it says:
Shaykh Muhammad ibn al-Shaykh Abd al-Lateef ibn Abd al-Rahmaan and Shaykh Sulaymaan ibn Sahmaan said: With regard to the mosque of al-Taa'if, on the northern side of which there is the grave of Ibn Abbaas (may Allah be pleased with him), it is permissible to pray in the mosque, if a high wall has been placed between the grave and the mosque, which means that the grave is outside of the mosque boundary. So it is not makrooh to pray there. End quote.
Others stipulated that the wall of the mosque should be high so that the graves cannot be seen by the worshippers. Shaykh Muhammad ibn Saalih al-Uthaymeen (may Allah have mercy on him) said:
If a person prays in a mosque in front of which is a graveyard, if there is a street - for example - or a full wall, so that the worshippers cannot see the graves, there is nothing wrong with that. But if it is close and directly next to the mosque, and there is no wall, or there is a wall but it is low and the worshippers can see these graves, then it is not permissible, because the Prophet (blessings and peace of Allah be upon him) said: Do not pray towards graves and do not sit on them. Narrated by Muslim
Liqa' al-baab al-Maftooh, 137; first question
It does not seem that what is mentioned above - about the wall being high and there being empty space in between - is binding, but if that is done then it is better and doing it is on the safe side. Rather it is sufficient if the mosque and the graves are separated by a wall which shows that the mosque is separate from the graves. One of the signs of them being completely separate is that each of them - the graveyard and the mosque - was built on its own land, and that the mosque was not built because of the graves. If that is the case, then it is permissible to pray in the mosque.
The scholars of the Standing Committee were asked:
Some mosques and places where the Eid prayers and prayers for rain (istisqa') are offered are built near graveyards in such a way that the graveyard is in the qiblah of the mosque and is separated from it by only a few metres. Some of the mosques are right next to the graveyards, and places where the Eid prayers and prayers for rain (istisqa') are offered are surrounded with a wall that separates them from the graveyard, whilst some others do not have a wall separating them from the graveyard. What is the ruling?
If these mosques and places where the Eid prayers and prayers for rain (istisqa') are offered were not built near the graveyards because of the graveyards or to honour those who were buried in them or to seek more reward for praying in them because they are close to the graveyards, then building them for praying and drawing closer to Allah in them is permissible, praying in them is acceptable and frequenting them for the purpose of prayer and other acts of worship that may be done there is something that is encouraged in Islam. If the graveyards are surrounded with walls that separate them from the mosques, that is sufficient. With regard to those that are not surrounded with walls, walls should be built to separate them from the mosques and places where the Eid prayers and prayers for rain (istisqa') are offered. If it is possible to leave empty space between the walls of the mosques and places where the Eid prayers and prayers for rain (istisqa') are offered, and the walls of the graveyards, that is more on the safe side. But if the mosques were built around the graveyards in order to venerate the graves, it is not permissible to pray in them and they should be demolished, because building them in the manner mentioned is one of the things that lead to shirk, i.e., associating the occupants of the grave with Allah. And it is narrated in a saheeh report that the Prophet (blessings and peace of Allah be upon him) said: Do not pray towards graves or sit on them. It is also narrated from him in a saheeh report that he said: Those who came before you used to take the graves of their Prophets and righteous men as places of worship; do not take graves as places of worship for I forbid you to do that. Both hadeeths were narrated by Muslim in his Saheeh; the hadeeths concerning this matter are well known.
Shaykh Abd al-Azeez ibn Baaz, Shaykh Abd al-Razzaaq Afeefi, Shaykh Abd-Allah ibn Ghadyaan, Shaykh Abd-Allah ibn Qaood
Majallat al-Buhooth al-Islamiyyah, 15/78, 79
See also the answer to question no. 13490
And Allah knows best.
Reproduced from Islam QA