3.1 Basic distributions

From now on, if you’re completely new to the tidyverse or to R itself, things might get shadier in terms of code, but I promise the visualizations will still be worth!

There are some variables that are not obligatory in the registration form for the R4DS Online Learning Community. That said, we shoul firstly have a look at NA values in our dataset.

#Counting NA cases in each column
map_dbl(dados, ~sum(is.na(.) | . == ""))

##     Type  RStudio    Stats Location   Cohort      lon      lat 
##        0        0        0        0        0      122      122

#Removing rows with only NA cases
dados <- dados %>% filter(!is.na(Type))

Well, looks like there are 3 rows with no data at all and other 4 rows with no location. I removed rows with NA values in every column (no data at all). I didn’t remove the rows with no location because they’ll still contribute with other information and we can remove them right before using the Location column.

Now that we’ve already cleaned the dataset from useless rows let’s have a look at each variable separately and work from that! The plot tabs below show frequency distributions for each variable. It is worth to note that each tab in the tabset will display a different code, just as it displays a different graph.

##{.tabset .tabset-fade .tabset-pills}

#Transforming the data and generating the plot
plot <- dados %>% 
  count(Cohort) %>%
  mutate(Count = n, Proportion = round(n/sum(n)*100, 2)) %>%
  ggplot(aes(Cohort, Count, labels = Proportion)) +
  geom_bar(fill = "dodgerblue2", stat = "identity") +
  labs(title = "Members' frequency by Cohort", y = "Frequency")

#Ploting the interactive graph centralized
div(ggplotly(plot), align = "center")

#Transforming the data and generating the plot
plot <- dados %>% 
  count(Type) %>%
  mutate(Count = n, Proportion = round(n/sum(n)*100, 2)) %>%
  ggplot(aes(Type, Count, labels = Proportion)) +
  geom_bar(fill = "dodgerblue2", stat = "identity") +
  labs(title = "Members' frequency by Type", y = "Frequency")

#Ploting the interactive graph centralized
div(ggplotly(plot), align = "center")

#Transforming the data and generating the plot
plot <- dados %>% 
  count(RStudio) %>%
  mutate(Count = n, Proportion = round(n/sum(n)*100, 2)) %>%
  ggplot(aes(RStudio, Count, labels = Proportion)) +
  geom_bar(fill = "dodgerblue2", stat = "identity") +
  labs(title = "Members' frequency by confortability with RStudio", y = "Frequency")

#Ploting the interactive graph centralized
div(ggplotly(plot), align = "center")

#Transforming the data and generating the plot
plot <- dados %>% 
  count(Stats) %>%
  mutate(Count = n, Proportion = round(n/sum(n)*100, 2)) %>%
  ggplot(aes(Stats, Count, labels = Proportion)) +
  geom_bar(fill = "dodgerblue2", stat = "identity") +
  labs(title = "Members' frequency by confortability with Statistics and Analysis", y = "Frequency")

#Ploting the interactive graph centralized
div(ggplotly(plot), align = "center")

It is worth to note that these plots concern people who joined up, not necessarily active people. That said, the first plot shows us the frequency of each invitation (Cohort). We can also see at the Type frequency that there is a 3.8 Learner/Mentor ratio.

As for RStudio, it has mean 2.9 with the frequency distribution rather stable despite not very higher frequencies around 3. Such behavior is stronger in Stats which has steeper increases as we approach the scale 3 from each side with a mean of 3.2. We could say that our average member is a 3 in the comfortability both in terms of RStudio and Stats, however, in terms of RStudio there is more variability around 3 than Stats.

But before any thoughts about this info we should look deeper into the data, seeing if these frequencies change based on any variable’s interaction.

3.2 Vizualizing Relations

We’ve seen some basic frequency distribution, but are RStudio’s and Stats’ frequency distributions equal when we plot them separately for each Type?

The first two plots show us that there really are differences in frequency distributions when we look at each Type separately. Mentors tend to choose higher scales of comfortability both in RStudio and Stats.

However, it gets better when we look at the third graph, _Learners__ are quite frequent when the scales from 1 to 3. But there are a lot of configurations with people highly comfortable in working with RStudio, Stats or both! But now have a look at the Mentors’ plot: there are no Mentors who chose scales lower than 3 in BOTH RStudio and Statistics. And when there IS a 2, the other is a 4.

Well, interestingly expected pattern, right? I mean, usually, someone who takes the position of Mentor does it as a result of comfortability working with at least one: RStudio and Stats. As it is possible to see in the Mentor’s plot, most of them are pretty comfortable with both RStudio and Stats.

I would guess that most Mentors with lower scales chose this position for the challenge of improvement by helping others like I did!

To explore this data and perceive these patterns is cool, but the thing is: at the end of the day, everybody always help others to find a solution and to get better, no matter if it is a Mentor or a Learner. That why the R4DS Online Learning community is awesome!

What about NOW we see all these awesome members scattered on a map?

3.2 Members’ World Map

The best world maps are the interactive ones! We’re going to need some coordinates so we can plot the map, right? But first, do you remember those Location rows with NA values? We’re going to remove them now:

#Remembering how many of them are there
map_dbl(dados, ~sum(is.na(.)))

##     Type  RStudio    Stats Location   Cohort      lon      lat 
##        0        0        0        0        0      122      122

#Removing rows with NA in Location
mp_dt <- filter(dados, !is.na(Location))

And there they go, they served their purpose and now are going to Valhalla. I assigned the cleaned data to the mp_dt variable, which stands for “maps data”.

Now, the ggmap package provides us with the geocode() function, which receives a vector of locations, try to find them in Google Map’s database and returns the best guesses’ coordinates. We’re going to pass Location to the function:

#Passing locations to be searched in Google Maps
lc_dt <- geocode(tolower(mp_dt$Location), source = "google")

mp_dt <- cbind(mp_dt, lc_dt)

It is worth to note that ggmaps::geocode() doesn’t always return coordinates for all the locations. It happens for 2 reasons: (I) Since the locations are user-typed data, there might be incomplete data keeping geocode() from finding a fair enough answer; (II) At some times google will block some API calls returning “OVER_QUERY_LIMIT”.

After searching for a while about the (II) issue, I found out that it happens because of an intern query limit for the Google API’s Calls. This issue might be solved for some locations by running the code and storing the non-error results in a separate variable and then repeating the process to try and obtain some other locations. Like the code below.

#Run these 2 lines once
longitude <- vector("double", length(mp_dt$Location))
latitude <- vector("double", length(mp_dt$Location))

#Run these 3 lines as much as yout think is necessary for the result you want
lc_dt <- geocode(tolower(mp_dt$Location), source = "google")
longitude <- ifelse(is.na(lc_dt$lon), longitude, lc_dt$lon)
latitude <- ifelse(is.na(lc_dt$lat), latitude, lc_dt$lat)

#Check how many NAs are still there
sum(is.na(longitude)) #Count
mean(is.na(longitude)) #Percentage

#If you're satisfied with the results, bind the coordinates with the data and go on
mp_dt <- cbind(mp_dt, longitude, latitude)

I didn’t wrap the code in a while() loop to prevent infinite loops in case the errors never get to 0%, as it could happen in the (I) case.

In one of my recent Kernels as a newbie in Kaggle ( Terrorism Worldwide - Exploratory Analysis ) I wrote a not-that-good function to plot choropleth maps using the leaflet package. I’m going to use that function’s code and modify it a little in a way that it plots our members with labelled markers instead of countries’ polygons.

In order to plot a map using R, we need data concerning the map itself. The maps::map() function provides us with that.

#Obtaining world map polygons
world <- maps::map("world", fill = T, plot = F)

#Creating a leaflet basic map
m <- leaflet(world) %>% addTiles()

I used the world data to generate the basic layer with leaflet() and addTiles() from the leaflet package.

Also, since we’re intended to plot markers for each member, we could use different colors for Mentor and Learners to see it on the map.

#Creating categorical color palette
pal <- colorFactor( RColorBrewer::brewer.pal(2, "Dark2"), 
                   domain = mp_dt$Type, na.color = "white")

We’re going to generate some variable strings to be plotted in the map with the markers, these strings are generated with sprintf() and strings based on the C language.

#Generating texts
strings <- sprintf(paste("<strong>%s</strong><br/><strong>%s</strong><br/>", 
                         names(mp_dt[2]), ": %s<br/>",
                         names(mp_dt[3]), ": %s<br/>",
                         names(mp_dt[5]), ": %s"),
                   mp_dt[[4]], mp_dt[[1]], mp_dt[[2]], mp_dt[[3]], mp_dt[[5]])

#Converting it to html format
labels <- strings %>% lapply(htmltools::HTML)

Now we only have to assemble all the parts in the base layer to have our map! I used leaflet::addCircleMarkers() to represent each member with a circle filled with colors based on Type. For guidance with the colors I added a legend with leaflet::addLegend(), and now our interactive map is complete!

#Adding polygon with the variable
m %>% 
  addCircleMarkers(lng = mp_dt$lon, 
                      lat = mp_dt$lat, 
                      label = ~labels, radius = 1, 
                      color = pal(mp_dt$Type), opacity = 0.75) %>%
  addLegend("bottomright", pal = pal, values = ~mp_dt$Type,
  title = names(mp_dt)[1],
  opacity = 10)

Every circle on the map is identified by all the varibales in the dataset! There are a lot of members on North America and Europe! South America and India come right after in the “rank”.