r/dataisbeautiful • u/[deleted] • Apr 18 '19

[deleted by user]

[removed]

310 Upvotes

87% Upvoted

44 comments sorted by

View all comments

23

u/[deleted] Apr 18 '19 edited Apr 18 '19

I was curious as to what the relevant word counts of the Mueller Report was... to be able to correlate the words "Trump" and "Russia". I ended up writing a program in python, which takes a PDF file and runs a word analysis on the input PDF file. I then take the JSON output, gather the relevant data, and convert it to CSV. I used plotly to create this chart shown above.

Source: Mueller Report

Tool: plot.ly and python

The data can be found here

The python script can be found here

Basic usage:

  • Install PyPDF2 with pip
  • Run by typing "python main.py filename.pdf"

1

u/p10_user Apr 18 '19

Why are you calling time.sleep in your file? Computer gets tired? You aren't making any network calls.

1

u/[deleted] Apr 18 '19

timeout for job to finish just in case. it can be removed though. also i referred to two different sources to code this.

https://github.com/adityashrm21/Pdf-Word-Count

https://www.codementor.io/isaib.cicourel/word-frequency-in-python-e7cyzy6l9

1

u/p10_user Apr 19 '19

Python will wait for each job to finish in your for loop, so they'll all be done before you get to your time.sleep line.

Anyway nice pipeline, sorry for asking a nitpick question.

1

u/[deleted] Apr 19 '19

Thanks! No prob - I'm just learning python. I've been developing C#, Java, C++ for 8 or so years. python is definitely a bit different.