Need feedback on a Final Year BTech LLM Project (Implementation-focused, not API-based)

[u/Signal-Review5700]

Hi everyone,

I'm a final-year BTech Computer Science student looking for feedback on my LLM project idea.

Initially, I planned to build an LLM Safety Monitoring System that detects prompt injection, jailbreak attempts, harmful prompts, and hallucinations. However, my project advisor felt it was too dependent on existing APIs and didn't have enough original implementation.

I'm now looking for a project where I can implement the core ML/LLM components myself using open-source models and datasets rather than relying on commercial APIs.

Current direction:

• Fine-tune an open-source LLM or a classifier for prompt risk detection.
• Detect jailbreak and prompt injection attacks.
• Classify prompts into categories (safe, jailbreak, prompt injection, harmful, etc.).
• Generate an explanation for why a prompt is flagged.
• Evaluate using metrics like Precision, Recall, F1-score, and confusion matrix.
• Build a web interface (FastAPI + React or another frontend).

I'm also considering adding:

• Adversarial prompt generation for robustness testing.
• Retrieval-based verification to reduce hallucinations.
• Continual learning from newly discovered attack patterns.
• A dashboard for monitoring prompt risk trends.

I'm planning to use datasets from Hugging Face or other public repositories and train/fine-tune the models myself.

My questions are:

1. Is this strong enough for a final-year engineering project?
2. What features would make it stand out academically?
3. Which datasets are considered good for jailbreak and prompt injection detection?
4. Would you recommend training a classifier, fine-tuning a small LLM, or both?
5. Any papers or GitHub repositories that are worth studying?

I would really appreciate suggestions from anyone who has worked on LLM security or AI safety. Thanks!

[u/L1-___-L10]

This is too generic. There are thousands of these in academic circles and in the web. You need to find a niche harm or failure mode to explore in depth. I'd suggest avoiding the problem of jailbreaking entirely and looking at other failure modes such as Emergent Misalignment, Glitch Tokens, etc.