pew.Rmd

# Pew Research Center (PEW) {-}

[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0) <img src='https://img.shields.io/badge/Tested%20Locally-Windows%20Laptop-brightgreen' alt='Local Testing Badge'>

Public opinion polling on U.S. Politics & Policy, Journalism & Media, Internet, Science & Tech, Religion & Public Life, Hispanic Trends, Global Attitudes & Trends, and Social & Demographic Trends.

* Generally one table per survey, with one row per sampled respondent.

* Complex samples generalizing to the noninstitutionalized adults in the nation(s) surveyed.

* Varying publication dates for both [American Trends Panel](https://www.pewresearch.org/our-methods/u-s-surveys/the-american-trends-panel/) surveys of the United States and also for [International Surveys](https://www.pewresearch.org/our-methods/international-surveys/). [National Public Opinion Reference Survey](https://www.pewresearch.org/methods/2021/05/03/how-pew-research-center-uses-its-national-public-opinion-reference-survey-npors/) released annually since 2020.

* Administered by the [Pew Research Center](http://www.pewresearch.org/).

---

## Recommended Reading {-}

Four Example Strengths & Limitations:

✔️ [US panel design and synchronized fielding periods reduce differential partisan nonresponse bias](https://www.pewresearch.org/methods/2024/07/08/how-pew-research-center-uses-its-national-public-opinion-reference-survey-npors/)

✔️ [Surveys across multinational, multiregional, and multicultural contexts follow best practices](https://wapor.org/wp-content/uploads/AAPOR-WAPOR-Task-Force-Report-on-Quality-in-Comparative-Surveys_Full-Report.pdf)

❌ [American Trends Panel sample selected from postal data missing 2% to 10% of the population](https://www.pewresearch.org/race-and-ethnicity/2024/09/27/voters-on-immigration-methodology/)

❌ [Most US studies only fielded in English and Spanish](https://www.pewresearch.org/science/2024/11/14/trust-in-science-2024-methodology/)

<br>

Three Example Findings:

1. [In 2024, nearly half of US teens say they are online almost constantly, up from 24% a decade earlier](https://www.pewresearch.org/internet/2024/12/12/teens-social-media-and-technology-2024/).

2. [Across 34 nations, a median of 64% of adults rate their country's economic situation poorly in 2024](https://www.pewresearch.org/short-reads/2024/06/07/economic-ratings-across-34-countries-are-more-negative-than-positive/).

3. [About three-quarters of U.S. adults said news organizations tend to favor one side in 2024](https://www.pewresearch.org/short-reads/2024/12/02/most-americans-continue-to-say-media-scrutiny-keeps-politicians-from-doing-things-they-shouldnt/).

<br>

Two Methodology Documents:

> [U.S. Surveys](https://www.pewresearch.org/our-methods/u-s-surveys/)

> Country Specific Methodology, for example the [2022 Global Attitudes Survey](https://www.pewresearch.org/methods/interactives/international-methodology/global-attitudes-survey/all-country/2022/)

<br>

One Haiku:

```{r}
# sock puppet pundit
# throws 'ssue, cites pew-laced news, sighs
# "unbutton your eyes!"
```

---

## Download, Import, Preparation {-}

1. Register for a Pew Research Center account at https://www.pewresearch.org/profile/registration/.

2. `DOWNLOAD THIS DATASET` at https://www.pewresearch.org/global/dataset/spring-2022-survey-data/.

3. Download the SPSS dataset `Pew-Research-Center-Global-Attitudes-Spring-2022-Survey-Data.zip`:

```{r eval = FALSE , results = "hide" }
library(haven)

pew_fn <-
	file.path( 
		path.expand( "~" ) , 
		"Pew Research Center Global Attitudes Spring 2022 Dataset.sav"
	)

pew_tbl <- read_sav( pew_fn )

pew_label <- lapply( pew_tbl , function( w ) attributes( w )[['label']] )

pew_labels <- lapply( pew_tbl , function( w ) attributes( w )[['labels']] )

pew_tbl <- zap_labels( pew_tbl )

pew_df <- data.frame( pew_tbl )

names( pew_df ) <- tolower( names( pew_df ) )
```

Collapse country-specific cluster and strata variables into two all-country cluster and strata variables:

```{r eval = FALSE , results = "hide" }
# create the constructed psu and strata variables from among the
# non-missing country-specific columns
pew_df[ , 'psu_constructed' ] <-
	apply(
		pew_df[ , grep( "^psu_" , names( pew_df ) ) ] , 
		1 ,
		function( w ) w[ which.min( is.na( w ) ) ]
	)
	
pew_df[ , 'stratum_constructed' ] <-
	apply(
		pew_df[ , grep( "^stratum_" , names( pew_df ) ) ] , 
		1 ,
		function( w ) w[ which.min( is.na( w ) ) ]
	)

# for countries without clustering variables, give every record a unique identifier for the psu..
pew_df[ is.na( pew_df[ , 'psu_constructed' ] ) , 'psu_constructed' ] <-
	rownames( pew_df[ is.na( pew_df[ , 'psu_constructed' ] ) , ] )

# ..and zeroes for the stratum
pew_df[ is.na( pew_df[ , 'stratum_constructed' ] ) , 'stratum_constructed' ] <- 0
```

### Save Locally \ {-}

Save the object at any point:

```{r eval = FALSE , results = "hide" }
# pew_fn <- file.path( path.expand( "~" ) , "PEW" , "this_file.rds" )
# saveRDS( pew_df , file = pew_fn , compress = FALSE )
```

Load the same object:

```{r eval = FALSE , results = "hide" }
# pew_df <- readRDS( pew_fn )
```

### Survey Design Definition {-}
Construct a complex sample survey design:

```{r eval = FALSE , results = "hide" }
library(survey)

options( survey.lonely.psu = "adjust" )

pew_design <- 
	svydesign(
		id = ~ psu_constructed ,
		strata = ~ interaction( country , stratum_constructed ) ,
		data = pew_df ,
		weights = ~ weight , 
		nest = TRUE
	)
```

### Variable Recoding {-}

Add new columns to the data set:
```{r eval = FALSE , results = "hide" }
pew_design <- 
	update( 
		pew_design , 

		one = 1 ,
		
		topcoded_respondent_age = ifelse( age >= 99 , NA , ifelse( age >= 97 , 97 , age ) ) ,

		human_rights_priority_with_china =
			ifelse( 
				china_humanrights_priority > 2 , 
				NA , 
				as.numeric( china_humanrights_priority == 1 )
			) ,
		
		favorable_unfavorable_one_to_four_us = ifelse( fav_us > 4 , NA , fav_us ) ,
		
		favorable_unfavorable_one_to_four_un = ifelse( fav_un > 4 , NA , fav_un ) ,

		country_name =
			factor( 
				country , 
				levels = as.integer( pew_labels[[ 'country' ]] ) , 
				labels = names( pew_labels[['country']] )
			) ,
			
		econ_sit = 
			factor( 
				econ_sit , 
				levels = 1:4 , 
				labels = c( 'Very good' , 'Somewhat good' , 'Somewhat bad' , 'Very bad' ) 
			)
	)
```

---

## Analysis Examples with the `survey` library \ {-}

### Unweighted Counts {-}

Count the unweighted number of records in the survey sample, overall and by groups:
```{r eval = FALSE , results = "hide" }
sum( weights( pew_design , "sampling" ) != 0 )

svyby( ~ one , ~ country_name , pew_design , unwtd.count )
```

### Weighted Counts {-}
Count the weighted size of the generalizable population, overall and by groups:
```{r eval = FALSE , results = "hide" }
svytotal( ~ one , pew_design )

svyby( ~ one , ~ country_name , pew_design , svytotal )
```

### Descriptive Statistics {-}

Calculate the mean (average) of a linear variable, overall and by groups:
```{r eval = FALSE , results = "hide" }
svymean( ~ topcoded_respondent_age , pew_design , na.rm = TRUE )

svyby( ~ topcoded_respondent_age , ~ country_name , pew_design , svymean , na.rm = TRUE )
```

Calculate the distribution of a categorical variable, overall and by groups:
```{r eval = FALSE , results = "hide" }
svymean( ~ econ_sit , pew_design , na.rm = TRUE )

svyby( ~ econ_sit , ~ country_name , pew_design , svymean , na.rm = TRUE )
```

Calculate the sum of a linear variable, overall and by groups:
```{r eval = FALSE , results = "hide" }
svytotal( ~ topcoded_respondent_age , pew_design , na.rm = TRUE )

svyby( ~ topcoded_respondent_age , ~ country_name , pew_design , svytotal , na.rm = TRUE )
```

Calculate the weighted sum of a categorical variable, overall and by groups:
```{r eval = FALSE , results = "hide" }
svytotal( ~ econ_sit , pew_design , na.rm = TRUE )

svyby( ~ econ_sit , ~ country_name , pew_design , svytotal , na.rm = TRUE )
```

Calculate the median (50th percentile) of a linear variable, overall and by groups:
```{r eval = FALSE , results = "hide" }
svyquantile( ~ topcoded_respondent_age , pew_design , 0.5 , na.rm = TRUE )

svyby( 
	~ topcoded_respondent_age , 
	~ country_name , 
	pew_design , 
	svyquantile , 
	0.5 ,
	ci = TRUE , na.rm = TRUE , na.rm.all = TRUE
)
```

Estimate a ratio:
```{r eval = FALSE , results = "hide" }
svyratio( 
	numerator = ~ favorable_unfavorable_one_to_four_un , 
	denominator = ~ favorable_unfavorable_one_to_four_us , 
	pew_design ,
	na.rm = TRUE
)
```

### Subsetting {-}

Restrict the survey design to :
```{r eval = FALSE , results = "hide" }
sub_pew_design <- subset( pew_design , country_name == 'South Korea' )
```
Calculate the mean (average) of this subset:
```{r eval = FALSE , results = "hide" }
svymean( ~ topcoded_respondent_age , sub_pew_design , na.rm = TRUE )
```

### Measures of Uncertainty {-}

Extract the coefficient, standard error, confidence interval, and coefficient of variation from any descriptive statistics function result, overall and by groups:
```{r eval = FALSE , results = "hide" }
this_result <- svymean( ~ topcoded_respondent_age , pew_design , na.rm = TRUE )

coef( this_result )
SE( this_result )
confint( this_result )
cv( this_result )

grouped_result <-
	svyby( 
		~ topcoded_respondent_age , 
		~ country_name , 
		pew_design , 
		svymean ,
		na.rm = TRUE 
	)
	
coef( grouped_result )
SE( grouped_result )
confint( grouped_result )
cv( grouped_result )
```

Calculate the degrees of freedom of any survey design object:
```{r eval = FALSE , results = "hide" }
degf( pew_design )
```

Calculate the complex sample survey-adjusted variance of any statistic:
```{r eval = FALSE , results = "hide" }
svyvar( ~ topcoded_respondent_age , pew_design , na.rm = TRUE )
```

Include the complex sample design effect in the result for a specific statistic:
```{r eval = FALSE , results = "hide" }
# SRS without replacement
svymean( ~ topcoded_respondent_age , pew_design , na.rm = TRUE , deff = TRUE )

# SRS with replacement
svymean( ~ topcoded_respondent_age , pew_design , na.rm = TRUE , deff = "replace" )
```

Compute confidence intervals for proportions using methods that may be more accurate near 0 and 1. See `?svyciprop` for alternatives:
```{r eval = FALSE , results = "hide" }
svyciprop( ~ human_rights_priority_with_china , pew_design ,
	method = "likelihood" , na.rm = TRUE )
```

### Regression Models and Tests of Association {-}

Perform a design-based t-test:
```{r eval = FALSE , results = "hide" }
svyttest( topcoded_respondent_age ~ human_rights_priority_with_china , pew_design )
```

Perform a chi-squared test of association for survey data:
```{r eval = FALSE , results = "hide" }
svychisq( 
	~ human_rights_priority_with_china + econ_sit , 
	pew_design 
)
```

Perform a survey-weighted generalized linear model:
```{r eval = FALSE , results = "hide" }
glm_result <- 
	svyglm( 
		topcoded_respondent_age ~ human_rights_priority_with_china + econ_sit , 
		pew_design 
	)

summary( glm_result )
```

---

## Replication Example {-}

This matches statistics and standard errors from [How to analyze Pew Research Center survey data in R](
https://medium.com/pew-research-center-decoded/how-to-analyze-pew-research-center-survey-data-in-r-f326df360713):

1. `DOWNLOAD THIS DATASET` at https://www.pewresearch.org/politics/dataset/april-2017-political-survey/.

2. Download the SPSS dataset `Apr17-public-4.3-update.zip` dated 12/29/2017:

```{r eval = FALSE , results = "hide" }
political_survey_2017_fn <- file.path( path.expand( "~" ) , "Apr17 public.sav" )

political_survey_2017_tbl <- read_sav( political_survey_2017_fn )

political_survey_2017_df <- data.frame( political_survey_2017_tbl )

names( political_survey_2017_df ) <- tolower( names( political_survey_2017_df ) )
```

Construct a complex sample survey design:

```{r eval = FALSE , results = "hide" }
political_survey_2017_design <-
	svydesign(
		~ 0 ,
		data = political_survey_2017_df ,
		weights = ~ weight
	)
```

Add new columns to the data set:
```{r eval = FALSE , results = "hide" }	
political_survey_2017_design <-
	update(
		political_survey_2017_design ,
		q1 = 
			factor( 
				q1 , 
				levels = c( 1 , 2 , 9 ) , 
				labels = c( 'Approve' , 'Disapprove' , 'DK/RF' ) 
			)
	)
```

Reproduce statistics and standard errors shown under `Estimating frequencies with survey weights`:

```{r eval = FALSE , results = "hide" }
result <- svymean( ~ q1 , political_survey_2017_design , na.rm = TRUE )

stopifnot( round( coef( result ) , 4 ) == c( 0.3940 , 0.5424 , 0.0636 ) )
stopifnot( round( SE( result ) , 4 ) == c( 0.0144 , 0.0147 , 0.0078 ) )
```

---

## Analysis Examples with `srvyr` \ {-}

The R `srvyr` library calculates summary statistics from survey data, such as the mean, total or quantile using [dplyr](https://github.com/tidyverse/dplyr/)-like syntax. [srvyr](https://github.com/gergness/srvyr) allows for the use of many verbs, such as `summarize`, `group_by`, and `mutate`, the convenience of pipe-able functions, the `tidyverse` style of non-standard evaluation and more consistent return types than the `survey` package. [This vignette](https://cran.r-project.org/web/packages/srvyr/vignettes/srvyr-vs-survey.html) details the available features. As a starting point for PEW users, this code replicates previously-presented examples:

```{r eval = FALSE , results = "hide" }
library(srvyr)
pew_srvyr_design <- as_survey( pew_design )
```
Calculate the mean (average) of a linear variable, overall and by groups:
```{r eval = FALSE , results = "hide" }
pew_srvyr_design %>%
	summarize( mean = survey_mean( topcoded_respondent_age , na.rm = TRUE ) )

pew_srvyr_design %>%
	group_by( country_name ) %>%
	summarize( mean = survey_mean( topcoded_respondent_age , na.rm = TRUE ) )
```