Primer 4 · The five dplyr verbs

Almost all data work is these five

If the cells below never become ready

This page runs R inside your browser. It needs about 25 MB on first load (the R engine, the machinery that runs and marks your code, the dplyr and readr packages, and the cohort file itself) and a minute or two on a slow connection. If nothing happens after that, your network is probably blocking it. Work through the R primer in RStudio instead. Everything taught here is taught there too.

There is no Stop button. If you write a loop that never ends, reload the page.

Almost everything you will do to a table on Day 1 is one of five verbs: keep some rows (filter()), keep some columns (select()), reorder rows (arrange()), add a column (mutate()), and collapse many rows into a summary (summarise()). Learn these five and you can do real work.

From here on you are working with the actual workshop cohort, the same 240 synthetic students the 11:15 session on Day 1 uses. By the time you sit down in the room, this file will already be familiar. For the full explanation of any of this, see the R primer.

Reading the data

read_csv() comes from readr. It takes a path to a file and gives you back a tibble. glimpse() from dplyr turns that tibble on its side so you can see every column at once, the right first move with any new dataset.

240 rows, 17 columns. Read the type tags down the left: <chr> is text, <dbl> is a number with decimals. Note that batch is a <dbl>: R read 2024 as a number, not as a label, so you write batch == 2024 without quotation marks.

The same line works unchanged in RStudio on Day 1, because your project will have a data/ folder holding the same file. That is the whole point of the data/meded_students.csv path: it is relative to your project, not to your computer, so the script runs on anyone’s machine.

One way this browser lies to you

The file arrived here by magic: this page downloaded it for you. In RStudio you will put the file in your project’s data/ folder yourself. The library() calls are the same magic in reverse: already run for you here, essential at the top of every script there. Type them anyway.

`filter()`: keep some rows

filter() keeps the rows where a condition is TRUE and drops the rest.

120 of the 240 students. Note the double equals sign: == asks a question, = assigns a value. Using = here is one of the two or three most common beginner errors in R.

`select()`: keep some columns

select() does for columns what filter() does for rows. Name the ones you want, in the order you want them.

Seventeen columns is more than you can read at once. Selecting five of them is often the difference between seeing your data and squinting at it.

`arrange()`: reorder the rows

arrange() sorts. It sorts ascending by default; wrap the column in desc() for descending.

Student S133 has the highest burnout score at 87.5. Sorting is how you find the extremes: the highest, the lowest, the implausible value that turns out to be a data-entry slip.

`mutate()`: add a column

mutate() adds a new column computed from existing ones. It never removes anything.

gain_check reproduces the gain column, which is a cheap way to confirm you have understood what a column means.

Look at row 3. posttest is NA (R’s marker for missing) and so gain_check is NA too. Arithmetic on an unknown value gives an unknown answer. Eleven students in this cohort have no posttest score, which is exactly what real data looks like, and it will bite you in Exercise 2.

`summarise()`: collapse rows into a summary

summarise() reduces many rows to one. Paired with group_by(), it reduces many rows to one per group, which is most of what a results table is.

Three batches, three rows. n() counts the rows in each group; always report it alongside a mean, so your reader knows whether that mean rests on 8 students or 80.

We spell it summarise. dplyr also accepts summarize, and the two are the same function, so use whichever spelling you prefer and be consistent.

Missing values: `NA` and `na.rm = TRUE`

That worked because every student has an attendance_pct. Try the same thing on posttest, where eleven students do not:

NA, not a number. One missing value is enough. R will not quietly decide which eleven students to drop on your behalf, so it hands the question back to you.

You answer it with na.rm = TRUE, which means remove the NAs first:

NA does not stop your script

An error stops. NA does not: it travels quietly through every calculation downstream and lands in your results table. Nothing warns you. When a number comes back as NA, a missing value upstream is almost always the reason.

Count them before you decide anything:

is.na() asks each value “are you missing?” and sum() counts the TRUEs. Dropping 11 of 240 students is a decision you should take deliberately, and report, not one that happens to you.

Putting them together

The verbs are designed to be chained. This is where Primer 3’s pipe pays off: read it as a sentence: take the students, keep the 2024 batch, keep four columns, sort by attendance, highest first.

Each verb does one small thing. The pipeline does something useful. Note that students itself is untouched: dplyr verbs return a new table rather than editing the old one, so you can experiment freely.

Exercise 1: two conditions at once

Time: 5 minutes

Keep only the students in the 2024 batch whose attendance_pct is above 85. Both conditions must hold.

students |>
  filter(batch == 2024, attendance_pct > 85)

Exercise 2: mean posttest by teaching arm

Time: 10 minutes

Produce one row per teaching_arm showing the number of students and their mean posttest score. Call the columns n and mean_posttest.

Remember the eleven missing posttest scores. mean() of anything containing an NA is NA. Try it without a guard first and see what comes back.

students |>
  group_by(teaching_arm) |>
  summarise(
    n = n(),
    mean_posttest = mean(posttest, na.rm = TRUE)
  )

Exercise 3: students per college

Time: 10 minutes

A stretch. Count how many students come from each college, with the commonest college first. There are two good routes: count(), a shortcut you have not met yet, or group_by() and summarise() followed by arrange(). Either is a correct answer.

students |>
  count(college, sort = TRUE)

# The long way round, using only the verbs above:
# students |>
#   group_by(college) |>
#   summarise(n = n()) |>
#   arrange(desc(n))

Check yourself

Four questions on the Moodle course

Take Check yourself · Primer 4 on the Moodle course. Four questions, unlimited attempts, no marks. Every answer comes back with an explanation, so a wrong one costs you nothing and teaches you something.

→ Primer 5 · Your first plot: turn that table into a picture.

Reading the data

filter(): keep some rows

select(): keep some columns

arrange(): reorder the rows

mutate(): add a column

summarise(): collapse rows into a summary

Missing values: NA and na.rm = TRUE