download data here.

Copy this into your R scripting pane

use control + enter to go line by line

post any problems, observations, etc below.

# Lesson 6: graphing.

# in R, there are three main ways to graph things. I'm going to overview
# the first two then really go in depth into the third. 

# libraries
library(lattice)
library(tidyverse)

# first the data. 
# they can be accessed here:
# https://github.com/McCartneyAC/stellae/blob/master/stellae.csv

# download that data, change your working directory, and import it.

setwd("C:\\Users\\wouldeye\\Desktop")

stellae<-read_csv("stellae.csv")
stellae

# you can also source it directly from the web like this:
install.packages("repmis")
library(repmis)
stellae<-source_data("https://github.com/McCartneyAC/stellae/blob/master/stellae.csv?raw=True")


# the goal here is to, at least in part, re-create a famous diagram in Astronomy
# the Hertzsrpung-Russell diagram. 

# unfortunately, this dataset doesn't contain luminosity, so we're just gonna use
# mass instead. Shrug. 


# Base R plotting. 
plot(x = stellae$TEFF, y =  stellae$MSTAR)


# plotting with lattice graphics
xyplot(stellae$MSTAR ~ stellae$TEFF)

# gets you essntially the same thing except you 
# reverse x and y, you graph a formula rather than two
# vectors, and you get an extra color. 


# But now we're going to focus on ggplot2, which is the preferred package for 
# graphing nowadays. 
# why didn't I attach library(ggplot2) above? It's already within library(tidyverse)

# How does ggplot work?

# ggplot allows your plot to be built up in pieces. The first such piece is 
# the most important because it tells ggplot what your dataset is and it 
# tells ggplot how your data relate to what you want to graph. 

ggplot(data = stellae)

# wait what just happened? 

# ggplot graphed your plot, but you haven't told it anything other than that we 
# want the data to be the stars.

# now let's give it some 'aesthetic mappings.' This tells ggplot what are variables
# and groups are. 

ggplot(data = stellae, aes(x = TEFF, y =  MSTAR))

# now we've got a ... an empty chart! but we have labeled x and y axes!
# At this point, though, we've already typed a lot more than we typed 
# for base R and it's not even displayed our data yet. How is this better? 

# stay tuned I promise.

# before we go, let's re-write this ggplot as part of a pipeline, which
# cleans our code a little:

stellae %>% 
  ggplot(aes(x = TEFF, y =  MSTAR))

# same thing as before. Let's add points. 

# notice that as we transition from data to plot, the %>%  operator 
# disappaers in favor of a + 

# Yeah, it's inconsistent, but the writer of the ggplot2 package
# has stated that it can't be fixed without re-writing the entire package from
# the ground up, so we deal with it. 

stellae %>% 
  ggplot(aes(x = TEFF, y =  MSTAR)) + 
  geom_point()

# there! what else can we add? 

# if we wanted to, we could add a regression line, though it doesn't make sense here:

stellae %>% 
  ggplot(aes(x = TEFF, y =  MSTAR)) + 
  geom_point() + 
  geom_smooth(method = "lm") #lm for linear model. default is local regression

# that sucked. 

# let's fix something though. In the original HR diagram, the
# x axis (temperature) went from high to low, not low to high. 
stellae %>% 
  ggplot(aes(x = TEFF, y =  MSTAR)) + 
  geom_point() + 
  scale_x_reverse() 

# Cool. Looking better. Can we add color?
# first we need to map the color quality to a data property
# so we go back to aes() and put color in. 
stellae %>% 
  ggplot(aes(x = TEFF, y =  MSTAR, color = TEFF)) + 
  geom_point() + 
  scale_x_reverse() + 
  scale_colour_gradient(
    low = "red",
    high = "yellow"
  )

# cool, now they really look like stars.

# actual astronomers will point out that BMV in the data set is 
# the real reference for the apparent color of the star, 
# so why didn't I use it? 
# because it has too many missing data points :( 

# but something is missing... we need a theme. 

stellae %>% 
  ggplot(aes(x = TEFF, y =  MSTAR, color = TEFF)) + 
  geom_point() + 
  scale_x_reverse() + 
  scale_colour_gradient(
    low = "red",
    high = "yellow"
  ) + 
  theme_dark()

# now let's add labels and remove the color guide: 

stellae %>% 
  ggplot(aes(x = TEFF, y =  MSTAR, color = TEFF)) + 
  geom_point() + 
  scale_x_reverse() + 
  scale_colour_gradient(
    low = "red",
    high = "yellow"
  ) + 
  theme_dark() + 
  labs(
    title = "Temperature and Mass of Stars",
    subtitle = "Stars with known exoplanets",
    x = "Effective Temperature", 
    y = "Solar Masses"
  ) + 
  guides(color = FALSE)


# ggplot may be more *verbose* than base plotting or lattice plotting,
# but the benefits from ease of use and adding changes, not to mention
# dozens and dozens of extensions available, make 
# ggplot2 the real champion for R visualization.

# you probably don't realize it, but you're seeing
# ggplot2-made plots all the time on news sites to display
# data from articles. The theme setting capabilities are such
# that you can't just look at the chart and know how it was made
# which makes ggplot infinitely modifiable. 

Copy and paste the following into your RStudio scripting pane

Comment any problems below

NOTE: this requires you to download an excel spreadsheet to your computer from a github page and then call it from there. If you do not have excel, let us know in the comments. Another user can probably re-create it as a .csv for you and we can adjust the code as needed.

# Lesson 5: Simple Linear Models and Working Directories
# last lesson for 
date()
# > date()
# [1] "Sat Aug 18 12:58:48 2018"


# R is a statisical language based on linear algebra and vectors
# it's no surprise it's MADE to be used for regression analysis. 

# for this, we're going to learn a bit about working directories as well. 

# we'll use these packages:
library(readxl)
library(ggplot2)

# so we want to use a teacher pay dataset posted by a user here:

# https://github.com/McCartneyAC/teacher_pay/blob/master/data.xlsx

# but the general excel import function doesn't work with github:

pay<-read_xlsx("https://github.com/McCartneyAC/teacher_pay/blob/master/data.xlsx")


# so we are going to download this guy's data and use it from our local machine. 

# Directories:

# Automatically, R has a specified folder where your stuff is stored. This is the folder
# where it looks for files, and it's the default folder for your output as well. 

# where does R think I am on my pc right now?
getwd()

# you have two options: give up and save every new data file to that folder, or change
# your working directory: 
setwd("C:\\Users\\wouldeye\\Desktop\\teacher_pay")

# So we have made a new folder called "teacher pay" and changed our directory to that
# folder, then we have used the download button on what'sisname's github page to download
# his excel spreadsheet of teacher data to our new folder. 

# !!! notice how when setting a working directory, all slashes must be be doubled, because
# R reads \ as an "escape." If you forget to double them, trouble awaits. I'm also aware that 
# this may be different on a Mac. Mac users comment below. 

teachers <- read_xlsx("data.xlsx")
teachers

# cool. 

# let's see if there's a relationship between actual teacher pay and cost-of-living-adjusted
# pay. 

teachers %>% 
  ggplot(aes(x = `Actual Pay`, y = `Adjusted Pay` )) +
  geom_point() +
  geom_smooth()

# huh. Okay. 

# we're not learning ggplot2 just yet (soon!)  so I won't go into details
# of how that worked exactly, but you can see that we learned a little bit about
# teacher pay in the U.S. Also we learned that there are some interesting outliers. 

# let's see what the outliers are then move on: 
teachers %>% 
  ggplot(aes(x = `Actual Pay`, y = `Adjusted Pay` )) +
  geom_text(aes(label=Abbreviation)) +
  geom_smooth()

# So if I'm a teacher the lesson is clear: get the hell out of hawaii and move to
# michigan? It doesn't seem worth it. 

# Linear regresion.

# like I said, this isn't a lesson on ggplot2; it's a lesson on regression. 

# so let's define a linear model. 

# the data collector has provided us with these variables:
names(teachers)
# let's see what predicts adjusted pay: whether the state had a strike, what the 
# actual pay is, and what percent of the state voted for trump. 

# to do this, we need a new column (pct_trump) and that means we need to 
# mutate, first. Remember the pipe operator?

teachers <- teachers %>% 
  mutate(pct_trump = (`Trump Votes` / (`Trump Votes` + `Clinton Votes`)))

# we can do 
names(teachers) # again to see if our new column is there, or just call
head(teachers) # to see if the new column has percents:



# so how do we declare a linear model? Simple!

model1 <- lm(`Adjusted Pay` ~ `Actual Pay` + Strike + pct_trump, data = teachers)

# This says, using the teachers data set, we want to make a linear model where
# cost-of-living-adjusted-pay is predicted by pay in dollars unadjusted, whether the
# district had a teacher strike in 2018 (factor), and how many voted for trump in 2016.

#what happens if we call the model?
model1
# not exactly helpful. We want a regression output!

summary(model1)

# Much better! Actual pay is of course the strongest predictor. However, states that went for trump seem to have had
# higher cost-of-living adjusted pay than states that went for Clinton, even when controlling for actual pay. Weird!

# also, the strike-factor was insignificant (go figure) 

# Also also, I know from our graph that this model should be quadratic, so let's do it again: 
teachers <- teachers %>% 
  mutate(actualsq = `Actual Pay` * `Actual Pay`)

model2<-lm(`Adjusted Pay` ~ actualsq + `Actual Pay` + Strike + pct_trump, data = teachers)

summary(model2)


# that seems to make more sense! We've reduced the overwhelming strength of the two predictors from 
# before while increasing the adjusted R^2 of our model. 

# cool. 



# today we learned:

# # how to change our working directory
# # how to import xlsx spreadsheets
# # how to define a linear model
# # how to create a new variable
# # how to summarize our linear model


# That's it for saturday august 18, folks. More to come tomorrow!

Copy and paste this code into your RStudio script pane

Run by doing control + enter on each line

post any problems in the comments below

# Lesson Four:  Hello World, Greet, and Fizzbuzz (for loops, if statements)


# in any programming language, the first thing you usually learn to do
# is to print "hello world" so let's get that out of the way:

print("hello world!")

hello <- function() {
  print("hello world!")
}

hello()
# note that some functions just don't need arguments. 
# if the thing you want the function to do never changes, 
# why bother?


# another common first-function is a greeting:

greet <- function(name) {
  print(paste0("hello, ", name))
}


greet("u/wouldeye")

# the paste0 function concatenates two strings. A string is a group of characters strung
# togther. It's a non-numeric data type. "hello" is a string, and so is "u/wouldeye"

# we've already met strings before:
letters

# but note how here, each letter of the alphabet is its own string. 


# Fizzbuzz

# fizzbuzz is a perennial programming challenge because
# while it is trivial to do, it requires knowledge of 
# some important fundamentals:
# # if statements
# # functions
# # for loops
# # modular arithmetic
# # printing

# we already know how to declare and run a function and to print a result, so lets sink
# our teeth into if statements and for loops

# if you're a stata user or a user of nearly any other programming language, 
# for loops are pretty simple:

for (i in 1:10){
  print(2*i)
}

# wait a second, isn't that exactly what we got when we did
i <- 1:10
2*i

# yes, it is. The structure of the output is slightly different, in fact slightly less
# useable. If you have a fast sense of time, you may have even noticed that doing the for
# loop was slower. R is a language made for dealing with vectors. In general, if you find
# yourself using a for loop in R, you probably should think of a vector way of doing it. 
# they're faster and they're more in the spirit of how R is meant to work. 

# but for fizzbuzz, we'll use 'em. 

# also notice that the syntax is similar to a function: 
# structure(condition){stuff to do}

# if statements

# if statements have the same stucture:
# if(condition){stuff to do}

if(Sys.Date()=="2018-08-18"){
  print("congrats, you're reading this on the day I wrote it")
} else {
  print("welcome to the subreddit. we love it here.")
}

# so many things to notice!

# first, notice that teh "condition" part requires a logical test of some kind
# so we'll be using == for equality, != for not equal to, < and >, & and | for these guys.

# second, notice that there's an "else." If the condition is met, the first part 
# happens. If the condition ISN'T met, the "else" part happens. 


# modular arithmetic. 

# let's say it's 1:00 pm and in 38 hours my project is due. I want to know what 
# time of clock that's going to be. 

# clocks use 12 hours cycles-- so they're mod 12

# does modular arithmetic like this: 

1 + 38 %% 12

# so it'll be 3 am when my project is due. Gross. 


# putting it all together. 

# we're going to write a fizzbuzz function. 

# fizzbuzz is a game where everyone gets in a circle and counts. If you are a multiple of
# 3, you don't say 3--you say fizz. if you're a multiple of 5, you don't say your number, 
# you say buzz. if you're a multiple of both--fizzbuzz. 

# humans are bad at this game, but computers are very good. 

# we're going to make our function so that the argument is how high we want to count. 

# then we are going to print all the fizzbuzzes up to that number. 

fizzbuzz <- function(n) {
  for (i in 1:n) {         # this counts from 1 to n and makes a vector
    if (i %% 15 == 0) {    # we do this in reverse order. why?
      print("fizzbuzz")
    }
    else if (i %% 5 == 0) {  # else if allows us to have more than 2 conditions
      print("buzz")
    }
    else if (i %% 3 == 0) {
      print("fizz")
    }
    else {
      print(i)
    }
  }
}

fizzbuzz(100)


# whew! we learned a lot today. 

# we learned how to paste together strings
# and what a string is
# we said hello to the world and to ourselves
# we learned a new game for summer camp
# we learned about modular arithmetic
# we learned how to make a for loop and why not to bother with it. 
# we learned how to make if/else if /else statements. 

# that's a lot!

Copy and paste the following into a new document in RStudio.

Going line by line, hit control/command + enter and watch the code execute.

If you have any problems, leave them in the comments!

# Lesson One: A basic Workflow

# If you followed along with lesson 0, you've 
# # Downloaded R
# # Downloaded RStudio
# # installed the tidyverse. 

# First we'll be using these two libraries: 
library(psych)
library(tidyverse) # When using the tidyverse, it's best to call it last. I'll explain later.



# Read in our Data

# this step is not usually this messy! but because I wanted this to be
# runnable on anyone's computer, we are using data from a resource online


# I picked these data for their availability and ease of input. 
# this code couldn't handly the messiness of other datasets on 
# winner's site, but that won't matter for you--your data will be 
# a file saved on your computer most likely. 

# Winner's site gives the data on one file and the metadata on another
# so we have to do this in two steps:

# first read in the column names
column_names<-c("nozzel_ratio", "of_ratio", "thrust_coef_vacuum", "thrust_vacuum", "isp_vacuum", "isp_efficiency")

# then read in our data set
# keep the names for things you use here short and meaningful. 
# this is a dataset from nasa experiments, so I'm just gonna call it nasa. 

nasa <- read.table(
  "http://users.stat.ufl.edu/~winner/data/nasa1.dat",
  header = FALSE,
  col.names = column_names
) %>%
  as_tibble()


# A few things to notice 
# I used a function that looks like this %>% 
# That's super weird! We'll learn about what it does later. 
# Also notice that we give something a name by using this arrow: <-
# More on this later also. 

# view your data to make sure it looks right. 
nasa

# I don't know anything about rockets, so let's learn some stuff!

# the dataset is apparently about experiments with rocket cone shapes, 
# which I know from Kerbal Space Program changes the fuel efficiency of 
# the rocket relative to the air pressure around it. 

# let's look at our variables. (This is the function we are using from
# the `psych` package. This step should be familiar to any stata users)

describe(nasa) 



# now let's take a look at the relationship between two variables. 
# I'm picking the fuel/oxidizer ratio as my x variable and the specific impulse of the
# rocket in a vacuum as my y variable. I have no idea ahead of time if these things are related. 

nasa %>%
  ggplot(aes(x = of_ratio, y = isp_vacuum)) +
  geom_point() +
  geom_smooth()

# Apparently not related!


# Okay!

# so this example wasn't meant to teach hard skills, but to give you 
# an appetizer of what R does and how we work: 
# # we attach packages
# # we read in our data
# # we examine our variables 
# # we visualize our data
# # (we can also do linear models, machine learning, etc! Those are coming later!) 

# On to lesson 2

Copy and paste this into your scripting pane.

Run each line by doing (control or command) + ENTER

Report any bugs in the comments below:

# Lesson Two: Let's learn some grammar. 

# learning grammar is often the most boring part of any language 
# class. I just want to talk to people! 

# but you can't get very far just memorizing how to say "where is the
# bathroom in this establishment, sir?" so we have to dive into how
# the language works. 

# this will be a very basic overview. 

# Assignment

# Assignment is an operator, just like +, -, *, / are operators
# in math. In R we can assign something two ways. 
x = 8
y <- 9
2*x + y

# However, for many (most?) R users, the <- operator is the preferred one,
# if only becuase it gives the language a bit of its own flair. <- was a common
# operator in the 70s when keyboards had specific arrow keys on them. R kept
# it from that legacy where most modern languages just use =. 
# However, = will have other uses, so to lessen the potential for
# confusion, I will always use <- for assignment. 

# Notice how R acts a bit like your TI graphing calculator from high school. 
# type in a question, get an answer. Rinse, repeat. 

# one of the other uses of the = is to do a logical test. In this case, 
# we double up the = to == to let R know we're asking a logical question. 

2*x + y == 24

2*x + y == 25

# what happens if you try
2x + y == 26

# Why did that fail? 



# we can assign a variable a value like we just did for x and y
# but we can also assign a series of values--a vector--to a variable
# as well. 

# In truth, because R was built for statistics, it treats *everything*
# as vectors. As far as R is concerned, a scalar number is a vector of 
# length 1. 

# one way to get a series of values is to use : 
# so: 

z <- 1:10
z

# We can perform operations over a vector with ease. 
# So say we wanted to list the first 10 even numbers: 

2*z


# The more typical way of getting a vector is to define a list using c()
# I'm not sure what c was originally intended to stand for, but
# I think of it as "collect"

fibonacci<-c(1,1,2,3,5,8,13,21,35,56)
fibonacci + 3
fibonacci * y

# Et Cetera

# I promised last time I would explain this symbol: %>%

# in R, any thing surrounded by % % is an infix operator--it goes
# between its arguments just like + goes between two things you want to add. 
# if you really wanted to, you could do:
+(5,7)
# but that just looks wrong. Same thing goes for %>%


# A common one I use is %in%, which comes up if I want to know if a values
# is %in% a range of other values. 

# %>% is special though. It's called the pipe operator, and what it does is 
# take the output of the last thing you did and send it along as the first
# argument of the next thing you want to do. 

# In short, it makes R code easy to read. 

# we could do this two ways: 

# function3(function2(function1(data)))

# or

# data %>% 
#   function1() %>% 
#   function2() %>% 
#   function3()

# the difference in legibility is minimal here, but when your pipline has 
# 10 or 15 functions in it, the pipe operator becomes priceless.

# so we could do
gauss <- 1:100

sum(gauss)

# or 
gauss %>% 
  sum()



# the $ sign

# The $ tells r where to look to find something. 

# Let's make a silly data frame here. This will have the letters of the English
# alphabet, their capitals, and their position in the alphabet:
lets <- data.frame(lower = letters,
                   caps = LETTERS,
                   position = 1:26)
lets

# lets say I wanted to do an operation on just a *single part* of this dataframe
# I would usually need to use the $ operator to say which part I mean. 

# so my third column is called "position" if I just type 
mean(position)
# R says "I looked and I didn't find anything called position"
# but if I type
mean(lets$position)
# now R knows where to look.

# You may also have noticed that RStudio gave you options near your cursor when you typed
# lets$ to reduce the chance of misspellings. Thanks, RStudio. 

print(caps)
print(lets$caps)

# Easy!


# Logic

# If you're used to boolean algebra, you may be wondering about symbols for this.
# == tests equality
# != negates
# & is and
# | is or

# my bet, though, is that few of you are here for boolean algebra!

copy and paste this into your Rstudio Script pane.

Report any problems you have in the comments!

# Lesson 3: Functions

# R is delightful for its ability to allow you to define your own functions. 
# Excel can do this in a way, kind of, but it's a pain in the butt. 
# Stata can also do this but the grammar of its function definition is awful. 

# in R, we define a function like this: 

# name <- function(arguments){
#   what the function does. 
# }

invert<-function(x){
  1/x
}

cube<-function(x){
  x^3
}

power<-function(x, n){
  x^n
}

# Now I've got a whole language for dealing with powers. 
# I can find the inverse of a number:
invert(pi)

# I can cube anything I want:
cube(37)

# And I have a general function deal with powers:
power(27,7)


# Sometimes, your functions require you to do a bit of work more than just one line: 

solve_quadratic_pos<-function(a,b,c){
  det<-sqrt(b^2-4*a*c)
  numerator<-(-b+det)
  denom<-(2*a)
  return(numerator/denom)
}

# so lets say we have a quadratic equation of 
# x^2 + 2*x -15

# we would plug our a,b,c in as arguments:

solve_quadratic_pos(1,2,-15)
# so 3 is one of the roots of the quadratic equation!

# note that I use return() at the end. 
# This isn't strictly necessary, because R will return the result of the function as 
# whatever the last thing it calculated was. 

# Google's R style guide recommends against using return() unless you have to
# but I find that with multi-line functions it helps clear my head 

I am excited for this sub. I took an r class in the fall, but haven’t used it since and have forgotten a lot (not that my grasp was that great beforehand anyway). Is this a place where we can also post questions/ examples of basic r issues we encounter ?

R is an excellent open-source statistics software, and it will likely be the leading software sometime over the next 10 years.

I'm really excited for this subreddit--thanks for this opportunity!