KoboldAI is one of the leading open-source front ends and back ends for LLM inference focused on chat and story writing.
2.5 Not good