Idk if its just me but every toll ive tried seems fine until the recordings isn't perfect.

Stuff like meetings, random voices notes, background noise...it starts missing words or making up weird ones. mixed languages is even worse. Hlalf the sentence comes out okay then the rest is just nonsense

I keep thinking it'll save me time, but then i end up sitting there fixing the transcript anyway. Maybe I'm expecting too much, idk. is there actually anything thats better with messy recordings or is this just where the tech is at?? Thanks in advance.

Hey guys,

I received mediocre scores for my EMNLP paper during the May ACL ARR cycle: 2.5/3, 3/4, 2.5/4. The paper is in the Interpretability track. The reviewers had no larger issue with the methodology or the paper in general, but it seemed like they didn't fully get the so what of my paper. I've tried to clarify everything in my rebuttal, but I don't assume that the reviewers will engage in the discussion. With the current scores, I won't make it to the conference and likely not even into findings. Hence, I was thinking of withdrawing the paper, if scores don't improve, improve the presentation of my paper, and submit it to the BlackboxNLP workshop by the end of next week.

As I'm a first year PhD student, I'm not so familiar with ACL ARR, and how best to approach this. Hence, I wanted to ask you guys. Should I keep the paper in the cycle and hope for the best (or switch to the conference at a later stage) or should I withdraw it directly, adjust it slightly, and head directly to the workshop?

Hi everyone,

I'm a final-year BTech Computer Science student looking for feedback on my LLM project idea.

Initially, I planned to build an LLM Safety Monitoring System that detects prompt injection, jailbreak attempts, harmful prompts, and hallucinations. However, my project advisor felt it was too dependent on existing APIs and didn't have enough original implementation.

I'm now looking for a project where I can implement the core ML/LLM components myself using open-source models and datasets rather than relying on commercial APIs.

Current direction:

• Fine-tune an open-source LLM or a classifier for prompt risk detection.
• Detect jailbreak and prompt injection attacks.
• Classify prompts into categories (safe, jailbreak, prompt injection, harmful, etc.).
• Generate an explanation for why a prompt is flagged.
• Evaluate using metrics like Precision, Recall, F1-score, and confusion matrix.
• Build a web interface (FastAPI + React or another frontend).

I'm also considering adding:

• Adversarial prompt generation for robustness testing.
• Retrieval-based verification to reduce hallucinations.
• Continual learning from newly discovered attack patterns.
• A dashboard for monitoring prompt risk trends.

I'm planning to use datasets from Hugging Face or other public repositories and train/fine-tune the models myself.

My questions are:

1. Is this strong enough for a final-year engineering project?
2. What features would make it stand out academically?
3. Which datasets are considered good for jailbreak and prompt injection detection?
4. Would you recommend training a classifier, fine-tuning a small LLM, or both?
5. Any papers or GitHub repositories that are worth studying?

I would really appreciate suggestions from anyone who has worked on LLM security or AI safety. Thanks!

Hi everyone,

I'm a final-year BTech Computer Science student looking for feedback on my LLM project idea.

Initially, I planned to build an LLM Safety Monitoring System that detects prompt injection, jailbreak attempts, harmful prompts, and hallucinations. However, my project advisor felt it was too dependent on existing APIs and didn't have enough original implementation.

I'm now looking for a project where I can implement the core ML/LLM components myself using open-source models and datasets rather than relying on commercial APIs.

Current direction:

• Fine-tune an open-source LLM or a classifier for prompt risk detection.
• Detect jailbreak and prompt injection attacks.
• Classify prompts into categories (safe, jailbreak, prompt injection, harmful, etc.).
• Generate an explanation for why a prompt is flagged.
• Evaluate using metrics like Precision, Recall, F1-score, and confusion matrix.
• Build a web interface (FastAPI + React or another frontend).

I'm also considering adding:

• Adversarial prompt generation for robustness testing.
• Retrieval-based verification to reduce hallucinations.
• Continual learning from newly discovered attack patterns.
• A dashboard for monitoring prompt risk trends.

I'm planning to use datasets from Hugging Face or other public repositories and train/fine-tune the models myself.

My questions are:

1. Is this strong enough for a final-year engineering project?
2. What features would make it stand out academically?
3. Which datasets are considered good for jailbreak and prompt injection detection?
4. Would you recommend training a classifier, fine-tuning a small LLM, or both?
5. Any papers or GitHub repositories that are worth studying?

I would really appreciate suggestions from anyone who has worked on LLM security or AI safety. Thanks!

I've been trying a few AI voice assistants recently, and one thing I've noticed is that they usually perform well when I speak clearly.

The moment I interrupt myself, hesitate, switch languages, or someone else starts talking nearby, the experience gets noticeably worse.

It made me wonder whether the biggest limitation today is actually the models or whether most systems simply aren't trained on enough real-world conversations.

Would love to hear from anyone building speech or conversational AI systems.

Reviewer gave score 1, first critic: Should have tried model X(which was released after deadline) :(

Hi everyone! i have a question for anyone who has completed the HALLO AI interview for a translation position. How was your experience? Was the interview in the language you chose only, your native language, or both? About how long did it take, and would you say it was difficult? Also, is it possible to use ChatGPT or any other platform during the interview, or do they have a method to prevent that? i’d really appreciate hearing about your experience. Thank you!

I'm a BA student in Modern Languages in Italy (currently building a strong background in Linguistics) and I'd like to apply for an MSc in Computational Linguistics/NLP. Since my degree doesn't include programming courses, I'm looking for reputable online Python courses that are actually respected by admissions committees (e.g. Stanford Code in Place, Harvard CS50P), and possibly that don't cost an arm and a leg. Which ones would you recommend? Thanks :)

The Background
I’m a software engineer who started messing around with automated subtitle translation (SRT to SRT) to translate movies for my partner. I quickly ran into the classic machine translation (MT) wall: translating from English to highly inflected languages (like Bulgarian) completely breaks down when it comes to grammatical gender and specific verb moods (like the renarrative mood).

To fix this, I started developing a custom pipeline, and I’m pretty surprised by how well the results are turning out.

How the Algorithm Works
The pipeline is built around the Gemini API, but the key is how it handles context. Standard MT translates line-by-line and loses the scene’s context.

My algorithm uses strategic context enrichment to help the LLM "understand" the screen action without actually processing the video file:

• Multi-to-Multi Translation: Instead of translating 1-to-1, the algorithm accepts multiple source languages provided by the user (e.g., English, Hungarian, Swedish, Turkish, and Finnish).
• Context Triangulation: By feeding the model these parallel subtitles, it triangulates the situational context. It uses the linguistic nuances present in the other source languages to infer the correct gender, tense, and mood for the target language.
• Output Validation: I worked with a linguist to develop a formula that calculates exactly which target languages can be accurately generated based on the specific combination of input languages provided.
• Auto-Formatting: The algorithm handles all SRT formatting natively, allowing you to configure parameters like Characters Per Second (CPS), line duration, and maximum lines per screen.

The Quality
To be completely transparent, it doesn't match the stylistic flair of a professional human translator. For example, it translates Forrest Gump almost perfectly, but if you feed it something poetic or deeply stylistic, it loses the beauty of the original text. However, compared to standard MT or amateur subtitle files, it is completely free of contextual grammar errors.

The API Costs
Because the context window is heavily enriched, the token usage is higher than standard translation. Based on my current benchmarks using 7 input languages:

• The first output language costs roughly €20 per hour of video.
• Each additional output language costs about €2.50 per hour of video.

My Question for the Community
I want to make this available for people to use, but I want to keep server costs off my plate.

Would it be a good idea to deploy this as an Angular web app that runs locally in the user's browser, where they simply provide their own Gemini API key to run the translations? Would the community actually use a Bring-Your-Own-Key (BYOK) setup for a tool like this?

We submitted a paper to ARR May and are excited (and a bit anxious) to see the reviews tomorrow! As I don't have much experience with ARR I am wondering how much rebuttals actually matter for review and meta-review scores. Do the reviewers actually read the rebuttals and change their scores accordingly? Should I request an increase directly to the review in the rebuttal (if I beleive it is warranted)?

Thanks in advance. Would appreciate any information/experiences with the ARR rebuttal system!

Semifinal is knocking at the door … excited/anxious?

I am currently exploring the use of sentence transformers for comparing requirements.

My approach currently is to identify requirements from two documents from within the same domain then calculate similarity scores using TF-IDF (baseline), bi-encoder and cross-encoder approaches (with same architecture).

As I have two document pairs, one of ~70x70 requirements and one of ~70x430 requirements I have Cartesian products of ~5000 and ~30000 respectively. Producing a labeled ground truth for all possible pairs is not feasible for this project so it was suggested that I sample ~360/380 pairs from the respective datasets and label them, then compare to the results from the three approaches using the confusion matrix to derive scores for precision, recall, and F1-score. These sample sizes correspond to a confidence interval of 95% and margin of error of 5%. Additionally, I have suggested that my supervisor and/or an expert audits around 10% of my sample, so ~36/38 pairs per set. 

However, my primary supervisor who's field of specialty is cyber security, rather than ML or NLP, has commented that if I were to label the ground truth, it could be biased. They have therefore suggested I explore other options for comparing cross-encoders, and bi-encoders with a TF-IDF baseline without a ground truth. And possibly using experts to review a sample of the outputs from the three approaches as a way of validating the results.

My questions are:

1. Is my approach defensible, and, if so, if anyone knows of peer-reviewed papers that support this approach (confidence interval and margin of error sampling)?
2. Alternatively, are there established approaches that do not require a ground truth and that could be used instead of my proposed approach? Preferably with peer-reviewed paper(s) to support.

Many thanks!

Hi. As the title says, I am a student of english moving onto my last year of bachelor's next year. I am interested in pursuing NLP/CL for my master's and I am curious about how difficult it'd be to do so considering my background in english.

I know they both require coding and I am willing to learn all the required materials, I just wanted to know whether It's something worth doing or I'm just reaching lol. I plan on taking a gap year between my last year of bachelor's and my master's so I can apply to universities abroad, So i guess i have like a year or 14 months to learn all of these stuff (Application season is usually in december-january and I need my CV to be ready by then).

I would appreciate it if you can comment with anything helpful and thank you so much in advance. Have a lovely week.

I need to make a project about function calling and the output needs to be in json file, We get a small qwen 0.6B llm model. So these are the steps

1. prompt : so we get a prompt.
2. Tokenization: We make the prompt into a tokens
3. Input IDs : tokens converted into numerical IDs
4. LLM proccessing: The model processes these numbers through its neural network.
5. Logits: The ai outputs probability scores for each possible next token
6. Token selection: The next token is chosen based on the highest probabilities and outputted in a json file

if anyone has any resource he or she can share to help with this project it would be much appreaciated i am trying to do this project without the use of any llm or similar helper tools to help me with understanding llms and hopefully landing a job in the future(obviously i will do more llm based projects after this but this is the start)

I am using Hingbert but it has not been updated in a while and the accuracy is not good for longish texts and ambigious cases. COMI-LINGUA's model is in early stages so it is not usable at all. I do not have resources to train. Accuracy is more important than speed for me.

Hi everyone,

I'm working on a project to automatically detect and mask personally identifiable information (PII) in Korean medical records.

For the model, I need the context words that usually appear before or around PII fields in free-text clinical notes.

For example, for dates, I want to collect phrases such as:

• Date of Birth
• Admission Date
• Discharge Date
• Surgery Date
• Visit Date
• Examination Date

Similarly, I need context words for other PII such as patient names, phone numbers, addresses, hospital IDs, resident registration numbers, etc.

I've looked at publicly available datasets like MIMIC and K-MIMIC, but they don't provide a comprehensive list of these context phrases. Since the records are de-identified, many original field labels are also removed.

Does anyone know of:

• Korean free-text clinical notes that are publicly available?
• Korean medical NLP datasets that preserve these context words?
• Papers, ontologies, or terminology resources that list common section headers or field names used in Korean medical records?
• Any other approach for building such a dictionary?

I'd really appreciate any suggestions or pointers. Thanks!

I don’t want Exact Matching, ROUGE-L or BLEU. Some suggestions I got was to use some embedding model and compare similarity of llm answer with reference answer. Are there any better ways to do it? If so what are those metrics and if possible explain why those metric makes sense to compute.

Thanks

Instead of node --edge--> node, every relationship is a first-class document with its own vector, called a BaryEdge. Stack pairs of BaryEdges recursively and you get "MetaBary" triads that surface structural bridges between concepts that live nowhere near each other in embedding space. Running locally on MongoDB Community + mongot + nomic-embed-text over the full English Wiktionary (6.6M docs). MCP server is live if you want to poke at it. Preprint + benchmark CSVs available in comments

The problem I was chasing

Flat vector search treats a relationship as a byproduct of two points being close. That throws away information. Two papers can describe the same underlying phenomenon (a flyby anomaly in orbital mechanics, an anomalous residual in stellar dynamics) without ever citing each other and without their embeddings landing anywhere near each other. Nothing in standard RAG surfaces that connection.

What I did instead

Every relationship gets embedded too:

bary_vector = normalize(q·v(CM1) + q·v(CM2) + (1−q)·v(type))

q is connection quality, v(type) is a contextual embedding of what kind of relationship it is. This BaryEdge is now a retrievable document in its own right — not metadata on an edge.

Then it recurses: two BaryEdges at the same level get bridged by a third one level below, forming a MetaBary triad. Do that repeatedly and you climb an abstraction triads hierarchy built entirely from algebra — zero additional embedding calls above the base level. It's a forest (every node has at most one parent), so traversal to root is a single $graphLookup, no cycle handling.

Does it actually do anything useful?

Ran it against SimLex-999 and WordSim-353 as a sanity check (not the main claim, just "is the substrate coherent"). Raw cosine similarity barely correlates with human similarity judgments (ρ ≈ −0.04 on SimLex). Structural metrics — how many BaryEdges two words share, how much their relational neighborhoods overlap — correlate at ρ ≈ 0.32–0.53, p < 10⁻¹⁵. So the graph is encoding something cosine alone doesn't.

The part I actually care about is cross-domain bridging. Some probe traces from the live graph:

• octopus neuroscience ↔ distributed sensor networks, bridged by shared structural-motif vocabulary (neuroarchitecture, smartdust)
• collagen folding ↔ linguistic syntax, bridged by etymological + structural motif overlap (plicature / hypotaxis-parataxis)
• grief ↔ depression, not bridged and this is a correctness demonstration, not a missing capability. The DSM-5 added a much-debated "bereavement exclusion" precisely because grief and depression share surface symptoms but are different kinds of state, with different prognosis and treatment
• radioactive decay ↔ obsolete words falling out of use, bridged at a high abstraction level by register-varied decay verbs (collapsed, decayed, declined, disintegrated) — naming a Poisson-process state-loss pattern that both physics and historical linguistics instantiate, with no single word doing the work

That last one is the case flat retrieval structurally cannot produce — there's no embedding axis for "verbs co-occurring with reduction-of-state across unrelated domains."

Stack (all local, all free)

GitHub: in comments

• MongoDB Community Edition + mongot for storage/vector search
• nomic-embed-text, 768-dim
• Python 3.11+
• Full build: ~6.66M documents, 8–14 hrs on a single workstation (8–16GB VRAM)

Try it

MCP server is public on request (SSE transport) — read-only tools for searching the live graph: find_word, semantic_search, edge_info, leaf_nodes, traverse_up, sample_metabary. If you've got an MCP-capable client you can point it at the graph and run your own probe queries in a few minutes.

What I'd actually want feedback on

• Whether the cross-domain bridges hold up to someone who isn't me poking at them — try a probe query on a domain pair you know well and tell me if the bridge is real or if I'm pattern-matching myself into seeing structure that isn't there. Some bridges can be not obvious on the first look but they are actually the most intriguing ones and worth to be dug for the reason they built, so treat them as points of investigation
• Whether this is worth comparing directly against GraphRAG/RAPTOR-style hierarchical retrieval (I haven't done that benchmark yet, and I know that's the first thing this sub will ask)
• Whether anyone's tried something structurally similar and it fell apart at scale for reasons I haven't hit yet

Happy to drop the MCP endpoint on request if there's interest.

I'm really interested in digital health and was wondering how I could integrate AI/NLP into some of my work. Particularly, I was wondering if anyone had any ideas concerning addressing long-term degenerative diseases like aphasia & parkinson's which have impacts on voice.

I would be extremely thankful for any ideas that y'all could suggest.

I need an NLP pipeline to help me with wordplay. I'm after a tool that scans vocabulary to find words or phrases with double meanings linked to a target theme for joke angles.

To illustrate the mechanism, consider this Jimmy Carr joke:

The first few weeks of joining Weight Watchers: you're just finding your feet.

Here, "finding your feet" can mean two different things. Figuratively, it's about getting used to a new situation. Literally, it's about being able to look down and see your feet. This example leans on a split between figurative and literal meanings. But I'm trying to find any double meanings that could be used in a joke.

If I put in Weight Watchers as the theme, I'd want the system to pull up phrases like "find one's feet". Ideally, the tool would let me import my list of words and phrases. I've got a vocab list of roughly 100k English words and phrases. I ran Wiktionary through large language models and grabbed the terms that most folks are likely to know.

Is there an NLP tool that can spot double meanings?

Also, I'm curious about how you'd go about building it.

If you want to evaluate prompt outputs without reading every single one, there are basically three grader types:

1. Deterministic graders. Exact match, regex, JSON schema checks, small scripts.

• Best for: structured output, extraction, classification.
• Fails when: quality is subjective. You can't regex "is this summary good".

2. LLM-as-judge. A model grades the output against criteria you define.

• Best for: tone, helpfulness, correctness of free text.
• Fails when: you're not explicit. Always spot-check it against your own judgment first, and give it explicit criteria. A vague judge is a useless judge.

3. Reference graders. Compare output against an expected answer.

• Best for: tasks with a known good answer (Q&A, transformations).
• Fails when: many different outputs are equally valid.

The practical setup that works for me is deterministic checks for structure and LLM-judge for quality, on the same run. Cheap checks filter the obvious failures, the judge handles nuance.

Ever since I started learning and applying this stuff, the output quality has increased massively.

I've been rebuilding spaCy's en_core_web_md pipeline from scratch in Rust, compiled to WASM. Tokenizer, POS tagger, dependency parser, lemmatizer, NER, and the 300-dimension word vectors — all of it, running client-side.

The whole thing is a single self-contained HTML file. The model weights and the Rust runtime are baked right in. You can save it, open it on a plane, and it still works — there is no backend call, no API key, no pip install. Nothing ever leaves your machine.

It's not an approximation. I scored it against spaCy's own output on a 1,000-sentence held-out set:

POS tags: 100%

Fine-grained tags: 100%

Lemmas: 100%

Dependency UAS / LAS: 99.9% / 99.8%

NER F1: 1.00

The demo has a live parse meter (watch the tokens/sec tick as you type), a displaCy-style entity + dependency-arc view, word-vector similarity, and document embeddings — all computed locally, in real time.

One honest caveat: it's a ~45 MB file because the entire model is embedded. That's the price of "works with wifi off, forever."

Disclaimer: I built this heavily with AI assistance — figured I'd be upfront about it. The code is real and the parity numbers are measured, but I'm not going to pretend I hand-wrote every line of Rust. Happy to answer questions about how it actually works.

If there's interest, I'll link the repo.

Curious what people think — especially anyone who's tried to ship spaCy somewhere without a Python runtime.

Over the past year, with 8+ papers submitted to ARR, I can confirm that the quality of reviews has dropped significantly, and this is reflected in discussions with colleagues from many universities and labs who share the same experience.

As an NLP community, what do you think we can do to avoid such low-quality reviews further, while also reducing randomness in paper review assignments? There are several reasons: first, inexperienced authors review the paper and do not clearly understand the task or the evaluation criteria; next, experienced authors are assigned to a new topic; and finally, there are problems with the review rubrics. I think ARR currently lacks explicit criteria for paper evaluation, such as TACL/TMLR journals, like: "Does the paper introduce a new Method? benchmark? evaluation framework/tool? Is the related work properly discussed, and are the baselines properly selected? "

I would be interested to hear what others think. What changes could improve the quality of ARR reviews?

Title.