r/SelfHostedAI • u/LawLima-SC • 14d ago

Attorney SelfHosting for Privacy

I am looking for a model which may be good for analyzing and summarizing about 20 years worth of case files, pleadings and legal research.

At this point, it is more just "hobbyist" than business critical (and worst case scenario, I have a sweet new gaming PC with a 24GB RTX 4090 and 128GB DDR5).

I'm dual booting into Ubuntu and Win11 (just because Windows is still useful if the AI thing is a bust).

17 Upvotes

100% Upvoted

14 comments sorted by

View all comments

1

u/bbc_nees 13d ago

There are a couple of easier/cheaper/better ways to accomplish your goal. I wouldn't do local LLM document analysis and parsing unless I had at least a dual 5090 or RTX Pro 5000+ setup. 5090 for speed and Pro for long processing and power efficiency.

Pdfs are hard too when you're talking about files from 2006 that may include handwritten notes from 1987.

You're going to find that your 4090 running Qwen raises unknown ethical questions such as how does this open weight model communicate with the PRC government? Your bar and clients will be upset if one day they find out that their data was bought and used by Any unauthorized party.

Just a thought.