This tutorial makes use of the following R package(s): readxl, Hmisc

# Reading data files into R

Data files can be loaded from the R session’s working directory, from a directory structure relative to the working directory using the single dot . or double dot .. syntax, or (for some file types) directly from a website. The following sections will expose you to a mixture of data file environments. For a refresher on directory structures, review Understanding directory structures.

## Reading from a comma delimitted (CSV) file

A popular data file format (and one that has withstood the test of time) is the text file format where columns are separated by a tab, space or comma. In the following example, R reads a comma delimited file called ACS.csv into a data object called dat.

dat <- read.csv("ACS.csv", header=TRUE)

If the CSV file resides on a website, you can load the file directly from that site as follows:

dat <- read.csv("http://mgimond.github.io/ES218/Data/ACS.csv", header=TRUE)

Note that not all data file formats can be readily loaded directly from a website in a “read” function without additional lines of code. Examples are given in the next two sub-sections.

To read other text formats that use different delimiters invoke the command read.table() and define the type of delimiter using the sep= parameter. For example, to read a tab delimited data file called ACS.txt, run the command read.table("ACS.txt", sep="\t").

## Reading from a R data file

R has its own data file format–it’s usually saved using the .rds extension. To read a R data file, invoke the readRDS() function.

dat <- readRDS("ACS.rds")

As with a CSV file, you can load a RDS file straight from a website, however, you must first run the file through a decompressor before attempting to load it via readRDS. A built-in decompressor function called gzcon can be used for this purpose.

dat <- readRDS(gzcon(url("http://mgimond.github.io/ES218/Data/ACS.rds")))

The .rds file format is usually smaller than its text file counterpart and will therefore take up less storage space. The .rds file will also preserve data types and classes such as factors and dates eliminating the need to redefine data types after loading the file.

## Reading from an Excel file

A package that does a good job in importing Excel files is readxl. It recognizes most column formats defined by Excel including date formats. However, only one sheet can be loaded at a time. So if multiple Excel sheets are to be worked on, each sheet will need to be loaded into separate dataframe objects.

If you don’t have the readxllibrary installed, install the package as you would any other package via RStudio’s interface or in R using the following command:

install.packages("readxl")

In this example, we will load an Excel data sheet called Discharge which tabulates daily river water discharge. The sample file, Discharge_2004_2014.xlsx, can be downloaded here. The following chunk of code assumes that the Excel file is saved in a folder called Data/ inside the R session’s working directory.

library(readxl)
xl <- read_excel("./Data/Discharge_2004_2014.xlsx", sheet = "Discharge")

Note that the single dot . that precedes the Data/ folder name instructs R to look for the Data/ folder in the current working directory. A later example will show you how to instruct R to look for files and folders outside of the current working directory using two dots .. instead of one.

An advantage to using this package for loading Excel files is its ability to preserve data types–including date formatted columns! In the above example, the Excel file has a column called Date which stores the month/day/year data as a date object. We can check that the loaded xl object recognizes the Date column as a date data type:

str(xl)
Classes 'tbl_df', 'tbl' and 'data.frame':   3866 obs. of  3 variables:
$Date : POSIXct, format: "2004-06-01" "2004-06-02" "2004-06-03" "2004-06-04" ...$ Discharge: num  6170 6590 6210 7120 6990 6160 5570 4500 4940 4550 ...
\$ Code     : chr  "A" "A" "A" "A" ...

The Date column is defined as a POSIXct data type; this is the computer’s way of storing dates as the number of seconds since some internal reference date. We would therefore not need to convert the date column as would be the case if the date column was loaded from a CSV file. If such was the case, then the date column would most likely be loaded as a character or factor data type. A more in-depth discussion on date objects and their manipulation in R is covered in the next chapter.

Excel files can be loaded directly from the web using the following chunk of code:

web.file <- "http://mgimond.github.io/ES218/Data/Discharge_2004_2014.xlsx"
tmp      <- tempfile(fileext=".xlsx")
xl       <-  read_excel(tmp, sheet = "Discharge")

Instead of downloading the file into virtual memory, R needs to download the file into a temporary folder before it can open it. However, that temporary file my not be available in a later session, so you will probably need to reload the data if you launch a new R session.

## Importing data from proprietary data file formats

It’s usually recommended that a data file be stored as a CSV or tab delimited file format if compatibility across software platforms is desired. However, you might find yourself in a situation where you have no option but to import data stored in a proprietary format. This requires the use (and installation) of a package called Hmisc. The package will convert the following file formats: SAS (XPT format), SPSS (SAV format) and Stata (dta format). You can install the package on your computer as follows:

install.packages("Hmisc")

In this example, a SAS file of blood pressure from the CDC will be loaded into an object called dat (file documentation can be found here).

library(Hmisc)
dat <- sasxport.get("http://personal.colby.edu/personal/m/mgimond/R/Data/BPX_G.xpt")

Likewise, to import an SPSS file, use the spss.get() function; and to import a STATA file, use the stata.get() function.

# How to save R objects to data files

## Export to a CSV file

To export a data object called dat.sub as a comma delimited file in a folder called Data/ residing at a level above the R session’s working directory, run the following:

write.csv(dat.sub, "../Data/ACS_sub.csv")

Note that the two dots .. tells the function to get out of the current folder (i.e. moving up a directory) before proceeding down into the Data/ folder.

## Export to a Rds file

To export a data object called dat.sub to a .Rds (R) file format in a folder called Data/ residing at a level above the R session’s working directory, run the following:

saveRDS(dat.sub, "../Data/ACS_sub.rds")

# Saving an R session

You can save an entire R session (which includes all data objects) using the save function.

To save all objects, set the list= parameter to ls():

save(list = ls(), file = "../Data/ACS_all.Rdata")

To save only two R session objects–dat and dat.sub–to a file, pass the list of objects to the list= parameter:

save(list = c(dat, dat.sub), file = "../Data/ACS_subset.Rdata")

load("../Data/ACS_all.Rdata")