BYU-I Course Project Data - A random sample of 50% of males and females for each year of runners for all years of the New York City marathon where gender is recorded.

Data details

There are 239,094 rows and 20 columns. The data source¹ is used to create our data that is stored in our pins table. You can access this pin from a connection to posit.byui.edu using hathawayj/marathon_nyc.

This data is available to all.

Variable description

age The age of the runner
gender The gender of the runner (M/F)
chiptime The time in minutes for the runner
year The year of the marathon
marathon The name of the marathon
country The country where the marathon was held
finishers The number of finishers at the marathon

Variable summary

Variable type: numeric

skim_variable	n_missing	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
age	0	1.00	40.08	10.15	17.00	32.00	40.00	47.00	90.00	▃▇▃▁▁
split_half	912	1.00	129.01	25.06	59.10	111.68	126.30	143.35	374.95	▆▇▁▁▁
clocktime	25032	0.90	281.07	58.87	122.17	238.90	275.23	315.20	599.98	▂▇▃▁▁
chiptime	0	1.00	269.11	51.58	122.03	233.72	263.81	297.22	599.98	▂▇▂▁▁
year	0	1.00	2005.97	4.19	1999.00	2003.00	2006.00	2009.00	2013.00	▆▇▇▆▇
split_10k	1574	0.99	61.80	12.85	26.18	52.97	60.32	68.95	615.00	▇▁▁▁▁
split_30k	119023	0.50	184.63	35.23	87.23	159.98	180.45	205.17	460.58	▃▇▁▁▁
split_40k	119114	0.50	254.40	48.93	116.95	220.07	249.10	282.82	594.32	▂▇▂▁▁
finishers	0	1.00	38136.35	7002.49	23267.00	34391.00	37950.00	43570.00	50062.00	▁▃▇▂▃
meantime	0	1.00	269.22	4.73	263.33	265.20	268.26	269.00	277.74	▅▇▁▁▃
female	0	1.00	0.33	0.47	0.00	0.00	0.00	1.00	1.00	▇▁▁▁▅
us	0	1.00	1.00	0.00	1.00	1.00	1.00	1.00	1.00	▁▁▇▁▁
canada	0	1.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	▁▁▇▁▁
europe	0	1.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	▁▁▇▁▁
other	0	1.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	▁▁▇▁▁
age_gender	0	1.00	1.00	0.00	1.00	1.00	1.00	1.00	1.00	▁▁▇▁▁

Variable type: character

skim_variable	complete_rate	min	max	n_unique
gender	1	1	1	2
marathon	1	22	22	1
country	1	2	2	1
marathon2	1	27	27	13

Explore generating code using R

pacman::p_load(pins, tidyverse, downloader, fs, glue, rvest, googledrive, connectapi)


# Data is from master_marathon. 
sdrive <- shared_drive_find("byuids_data") # This will ask for authentication.
google_file <- drive_ls(sdrive)  |>
  filter(stringr::str_detect(name, "master_marathon"))
tempf <- tempfile()
drive_download(google_file, tempf)
dat <- read_csv(tempf)



# Wrangle
marathon_nyc <- dat %>%
  filter(marathon == "New York City Marathon", !is.na(gender)) %>%
  group_by(year, gender) %>%
  sample_frac(size = .5) %>%
  ungroup()



board <- board_connect()
pin_write(board, marathon_nyc, type = "parquet")

pin_name <- "marathon_nyc"
meta <- pin_meta(board, paste0("hathawayj/", pin_name))
client <- connect()
my_app <- content_item(client, meta$local$content_id)
set_vanity_url(my_app, paste0("data/", pin_name))

Access data

This data is available to all.

Direct Download: marathon_nyc.parquet

R and Python Download:

URL Connections:

For public data, any user can connect and read the data using pins::board_connect_url() in R.

library(pins)
url_data <- "https://posit.byui.edu/data/marathon_nyc/"
board_url <- board_connect_url(c("dat" = url_data))
dat <- pin_read(board_url, "dat")

Use this custom function in Python to have the data in a Pandas DataFrame.

import pandas as pd
import requests
from io import BytesIO

def read_url_pin(name):
  url = "https://posit.byui.edu/data/" + name + "/" + name + ".parquet"
  response = requests.get(url)
  if response.status_code == 200:
    parquet_content = BytesIO(response.content)
    pandas_dataframe = pd.read_parquet(parquet_content)
    return pandas_dataframe
  else:
    print(f"Failed to retrieve data. Status code: {response.status_code}")
    return None

# Example usage:
pandas_df = read_url_pin("marathon_nyc")

Authenticated Connection:

Our connect server is https://posit.byui.edu which you assign to your CONNECT_SERVER environment variable. You must create an API key and store it in your environment under CONNECT_API_KEY.

Read more about environment variables and the pins package to understand how these environment variables are stored and accessed in R and Python with pins.

library(pins)
board <- board_connect(auth = "auto")
dat <- pin_read(board, "hathawayj/marathon_nyc")

import os
from pins import board_rsconnect
from dotenv import load_dotenv
load_dotenv()
API_KEY = os.getenv('CONNECT_API_KEY')
SERVER = os.getenv('CONNECT_SERVER')

board = board_rsconnect(server_url=SERVER, api_key=API_KEY)
dat = board.pin_read("hathawayj/marathon_nyc")

Footnotes

http://faculty.chicagobooth.edu/george.wu/research/marathon/data.htm ↩︎