If my cousins (who recently graduated from Fayette county high schools) are to be believed then it is a culture problem. According to them, sports were not looked upon favorably by most students and if you were to play a sport, soccer was the one of choice.
Lafayette and Henry Clay (among others) have had very talented teams, ones with multiple D1 players sometimes, but they could never channel that talent properly. Frederick Douglas appears to have a good coach, good culture, and good facilities. It wouldn't surprise me at all if they became the destination school for football in Lexingt