Thomas Ptashnik is a Psychology PhD student interested in working on this with us. He is using the SOEP-Core data and familiar with SEM/Latent variable methods.
We have gained access to the relevant data
Here's the link to the Fehr appendix that contains the survey items they created (starting at Appendix B on page 33).\
These items correspond to the SOEP-IS surveys, which can be found here (use item names, like Q132, to search quickly
These links also mention that individuals with preexisting data access can apply for expanded access. I [Thomas] have access to SOEP-core version 36 (1984-2020 surveys),..
DR: Some interesting content (at a quick peek)
From 2017...
Q380: What you value in your work likerts ... includes "Having much influence" and "Socially responsible and important work"
Q160: Optimism/pessimism about the future
Q162: ... bunch of Likerts on "attitudes towards life and the future" (e.g., 'The options that I have in life are determined by social circumstances.')
From 2019
... they seem to collect genetic data
A proposed project
Does the Fehr/SOEP data provide valuable 'outcome measures' of EA and effective giving support?
I think we might see positive responses to the Fehr et al questions and donation choices as ‘necessary but not sufficient' for people to become effective givers or even EAs. If (especially in spite of the de-biasing) people still don’t support international redistribution, international orgs, and don’t opt to give from the lottery earnings to the global poor person … I think they are very unlikely to be susceptible to an EA or effective giving (e.g., GiveWell) appeal. (See further discussion and debate on this below).
(But, as a check on this, it might be good to try to ask these same questions on a sample of actual EA’s and effective givers, and a comparison group!. #surveyexperiments)
Two projects on the same data
I envision two related projects on the same data: 1. Building a 'portable' model for prediction to aid targeting and 2. Building a 'deeper' model to aid understanding
I’m hoping that looking for predictors of (or ‘coherent factors explaining’) these responses in the SOEP data would prove useful for organizations like GWWC to consider ‘which groups to target in doing outreach’ (and perhaps especially ‘which groups to rule out’)
I hope we can do a sort of ‘leak-proof validated predictive ML model for this’
perhaps especially relevant for the German/EU context
Thomas: After talking it over with some colleagues, I think this approach is our best bet in terms of developing something with practical utility that still has a chance of being published in an academic journal. This is not my area of expertise, but if I remember correctly you have some R code already written. So I should quickly be able to put something together.
2. An (exploratory model) to help understand key factors that might be driving EA-adjacent attitudes and behaviors, offering insight into ‘what drives people towards or away from this mindset’.
Here we could engage the richer set of SOEP variables and consider latent factors
Anonymous colleague; caveats on 'the two goals'
if one simply wants to target people for giving to some specific EA-aligned cause in terms of a donation. In that case of the hypothetical African Christian women are likely to give, and it doesn't matter so much how they get to that decision. Quite a different set of metrics is desired (the kind of things we are trying to get at) if one is trying to actually select/find 'effective altruists'[RT2] if one simply wants to target people for giving to some specific EA-aligned cause in terms of a donation. In that case of the hypothetical African Christian women are likely to give, and it doesn't matter so much how they get to that decision. Quite a different set of metrics is desired (the kind of things we are trying to get at) if one is trying to actually select/find 'effective altruists'
Red team
But I'm less sure about: ..."would prove useful for orgs like GWWC to consider ‘which groups to target in doing outreach’ (and perhaps especially ‘which groups to rule out’)"
[suppose] you measure something like 'interested in giving to people in poverty in Africa' (or, at best, cosmopolitanism), and you find that the people highest in this are [Classical music fans], but the people most interested in EA stuff are [Techno ravers]. I think there are lots of reasons why this might occur. It could be that interest in EA is a combination of cosmopolitanism + interest in maximising effectiveness, but differences in the latter swamp the former. (If so the reasoning would at least be along the right lines, but would potentially be very practically misleading to GWWC)...
But I think what could be going on could be even worse, i.e.:
The measures measure something like 'not being so parochial that you won't give to a non-German charity', which is (ex hypothes) a necessary condition, but so minimal it's not really informing us about the much more demanding thing
... it measures something more specific/narrow that may be orthogonal or even antagonistic to EA (e.g. interest in overseas charity/poverty specifically [even if it doesn't maximise effectiveness]). Thought experiment: how would a libertarian-leaning AI-safety concerned German EA respond to the questions?
[still, this] seems worthwhile... I'd just be very tentative about inferring anything about what GWWC should do etc
Red team analogy
(I think of this case as a bit like studying interest in Marxism by asking about whether people are interested in helping the poor (or some such) In one sense you might think of this as a necessary condition / people who don't have any concern for this are not likely to be interested in Marxism. OTOH you'll probably mostly be picking up the 99% of people who are interested in helping the poor but not interested in the much more niche / slightly weirder thing that is also closely related to helping the poor, but is also associated with slightly counterintuitive views like 'donating to the poor is not good, you need to be concerned with [systemic change and global revolution / AI safety] etc.)
Red team:
[red team]
I guess it will be interesting to find out through your analysis:
Are these measures predicted by plain altruism + cosmopolitanism (which a priori we might say are more likely to be connected to EA)
Or are these measures predicted by egalitarianism + belief we should repay the third world / belief the rich should help the poor (which seem like they may be less closely connected with EA)*
*of course EAs are overwhelmingly liberal/egalitarian, but liberal/egalitarians are overwhelmingly not EA, which I think is an important complication"
DR and TP response to red team
Good points, and I even think “global redistribution” might rub some actual EAs the wrong way, as well as many EAs rejecting the 'repay our collective guilt' aspect.
Still, GWWC and TLYCS are pushing more for behaviors (esp. giving) than for intellectual alignment with EA. They are also pushing the traditional global poverty part of the EA agenda. I suspect the Fehr/Soep measures will pick up people more receptive to this than to longtermist 'avant garde' EA.
Thomas: This is the main point to highlight. We probably need to limit our generalizability to the people-oriented neartermist worldview bucket. As the comments above note, I'm not sure this worldview necessarily maps onto the longtermist individual concerned about, say, AI safety risk. However, as you point out, there is still utility in focusing on understanding individuals that have this worldview for GWWC and other EA orgs, and this worldview (according to the EA survey) is currently the largest in the community.
DR: Agreed, but we probably need to make sure not to water it down too much; ideally we would retain some notion of 'the importance of prioritization and cost-effectiveness' in the worldview we are targeting
TP: As you mentioned, it would be interesting to replicate this survey with explicitly EA endorsing individuals. Particularly, in seeing how well the ML model can predict cohorts that fall into the three different worldview buckets.
DR: yes but the model that predicts "EA/global poverty supporting types within a general population may be unlikely to predict groups *among explicit EA's*" ... still, the comparison could be interesting (and we've done a bit of this already with the EA survey)
TP: Also, as a long-term idea, it could be useful to consider developing more EA-oriented items for SOEP-IS (the survey Fehr and colleagues used) that take into account all the issues listed here.
DR: that would be great!
RT2: Is there any way you can think of to get at EA more like a style of thinking/justification of choices as opposed to possibly the highly context-dependent choices are themselves? Some kind of relevant psychometric things are probably possible e.g., need for cognition or something similar
RT1:
One option create or use measures of maximising + cosmopolitanism + altruism (or of maximising cosmopolitan altruism) ... maybe we are getting at 'EA style of thinking'. And if we can show that these more abstract measures are connected to behavioural or otherwise more concrete measures of EA inclination (whether that's decisions/choices, signing up for mailing list or something else) then it does seem reasonable to think of these as capturing EA inclination.
The risk otherwise is that theoretically we think these 3 things correspond to EA thinking... and actually they don't ...
Consider NFC, IRT, Rationality Quotient etc. as predictors of EA-inclination \
DR: My conception was maximizing + cosmopolitanism + _altruism + willing-to-sacrifice/non-competitiveness …_I think many people think “I should work to help humanity” but also think ‘yeah but I’ll be a sucker if I give to charity while my neighbor gets a new swimming pool and Hawaii holiday…’That’s where “willing-to-sacrifice/non-competitiveness” comes in, in my mind. (It needs a better name?)I think this last trait more important for effective giving than for EA-intellectual-engagement… and it may not be important at all for the latter.
Thomas: In psychology, altruism captures this notion. Prosociality is a concept of helping others but allows for self-concern, while altruism is distinguished by a purer form of selflessness (I have a paper under review that goes into detail about this, which I can privately share....argh, the closed doors of academia). Fortunately, altruism is widely studied and there are even a few items that capture it in the SOEP dataset.
\
Value of incentivized measures here
(DR ideas)
IMO it would be nice to have some meaningful behavioral (incentivized) measures on top of the ‘psych’ ones. The ‘donation to the very poor’ measure in Fehr et al gets at this a bit … although its a pretty small probablistic sacrifice. And I suspect it measures all three of the above except maximizing. And I don’t think these things are all separable, so I think that the fact that it measures ‘altruism and willing to sacrifice in a cosmopolitan-relevant context’ is good.
It would also be pretty nice to have a behavioral/incentivized measure of ‘maximizing in an altruistic context’ …If Fehr ea had asked them to (e.g.) allocate giving among a German poor person, an African poor person, and themselves, this might have been a decent measure.
(We have this choice in some other contexts though … not as rich data but maybe worth digging into). Why might that choice have been better (in some ways) than a hypothetical choice? Because I imagine in a hypothetical choice some people would be like “OK they obviously want me to say support the poor person in Africa, and I see the maximization arguments, so, fine.'But when it involves real money, and even their own money, I expect that for some people, other motives will outweigh the ‘maximizing motive’…“wait, I’d rather keep the money than give it to an African who will waste it”“wait, if this is real, I’d rather help someone local”.
Analysis Plan, sample, and variables under consideration (01/31/22, Ptashnik)
DR: See sidebar comments
Analysis plan
Lasso regression to identify the most salient cluster [DR: how is this defined?] of predictors for effective giving
I will use k-fold cross-validation to compare a lasso model with ridge regression and OLS to confirm it is the best method for handling our data [DR: 'best in what sense? I recommend the elastic net approach if possible.]
Bayesian and latent lasso
TP: There is now a Bayesian form of lasso, but the R packages to run this analysis are in their infancy and the results between the methods are strikingly similar (Steorts, 2015). So, on the first pass I will just use one of the methods above but may rerun the analysis time-permitting to check my assumption that results won’t change.
Similarly, there is latent lasso regression, but most of our constructs have only one indicator and the R package for this analysis also appears to be at a nascent stage.·
Sample
To start, I’m just considering the 2017 survey and the control group (i.e., those who weren’t notified of their position in the national and global income distribution (~700 individuals). We can expand to the 2018 survey and the treatment group in future analyses using the same method (although some items may not be included across surveys).
Outcome Variable
Q280 and 281 in the SOEP-IS dataset developed by Fehr et al. (2019)
You were paired with another household in Kenya or Uganda. This household belongs to the poorest 10 percent of households worldwide. Now, you have 50 EUR at your disposal and can split this amount between the other household and you in any way you want. If this task is selected for payout, you will receive the amount you decided to keep at the end of the interview. The amount you want to give the other household will be given in full to the other household (without transaction costs) at the end of the field period by Heidelberg University via a charitable organization. In full means that every given euro will be received by the other household 1:1. A leaflet with information about the donations will be given to you after you have made your decision. I ask you to make this decision alone now.”
“How much of the 50 EUR do you want to keep and how much do you want to give the other household?”
Below I list variables below in terms of what the intended construct I’m trying to get at and the proxy measures that are available within the SOEP dataset.
Theoretical rationale for construct from 'charitable giving' review
Theoretical rationale for these constructs comes from the most comprehensive review on predictors of charitable giving I could find (Bekkers & Wiepking, 2007; also see Bekkers & Wiepking, 2011 and Wiepking & Bekkers, 2012 for follow-ups on this review). These reviews seem like a reasonable starting point because they are cross-disciplinary and only consider studies that involve real money to real charitable organizations. There were a surprising number of what I think of as common-sense variables that weren’t included in these reviews that I add in the table below (i.e., those without an asterix).
There were several variables omitted because I did not think they were relevant or other constructs exist that better get at the underlying effect. ...
Home ownership: Appears to just be an indicator of wealth, so using income is preferrable.
Perceived financial position: Bivariate studies (Bennet & Kottasz; Havens et al., 2007) conclude those who perceive their financial situation as more positive are more generous donors. However, Fehr et al. (2019)—which has a more robust design—reports that “we find no evidence that perceived rank in the global income distribution affects support for global redistribution, donations to the global poor, globalization or immigration. If anything, when thinking about these policy preferences, it matters more how one compares to other people nationally than to others around the globe.” Given these findings and the fact that we are using the same data, it is probably sensible to omit this variable. Although studies have found confidence in the economy (Okunade, 1996), so an interesting pivot could be to measure optimism (both domain-specific and general forms).
Place of residence and years of residence: Mixed findings and it appears to be a weak predictor regardless.
Immigration and citizenship status: Better captured by other variables. “Osili and Du (2005) found that immigrants in the United States are less likely to give to charitable organizations and also give less, but that these differences are due to differences in racial background, lower levels of income, and education” (Bekkers & Wiepking, 2007: 15).
Youth participation: Impacts donations through socialization, which is better captured through parental background. It also strengthens social bonds of the children in the community, making them less likely to make effective donations over local causes.
Volunteering: In simple bivariate analysis, volunteers are usually found to donate more to charity. However, differences between volunteers and non-volunteers often vanish in multiple regression analyses controlling for joint determinants of giving and volunteering (Bekkers, 2002, Bekkers, 2006a, Wiepking & Maas, 2006). Given SOEP only asks about time spent volunteering and does not categorize where one volunteers, this variable seems like a blunt tool that is likely to be insignificant.
Awareness of need: A strong predictor of general philanthropy, but Fehr et al. (2019) did not find significant effects for effective giving.
DR: I think 'failing to find significant effects' shouldn't be reason to exclude this!
[DR: I think 'previous failire to find significant effects' shouldn't be reason to exclude!]
Variables held constant by the survey design (see Bekkers & Wiepking 2007 for detailed explanation): Solicitation, benefits, reputation, and efficacy.
DR comments:
A very interesting list of features
were these all asked before the charity questions? (I'm worried about reverse causality otherwise)
maybe remove 'unavailable' rows for space\
We should discuss how the fitted model will be used and interpreted ... maybe identifying a few collections of useful subsets:
One of the most studied variables in philanthropic studies. However, a large body of research finds that religious involvement is not related (or even inversely related) to secular giving (Brooks, 2005; Lyons & Nivison-Smith, 2006; Lyons & Passey, 2005). Still, given its prominence (and that fact that there are religious EA groups), it is worth including in our analysis.
Has been found to have a positive relationship with secular giving (Yen, 2002), more EA-aligned giving (e.g., development aid versus emergency aid; Srnka et al., 2003), and there are conflicting results on whether education impacts the amount donated (c.f., Schervish & Havens, 1997; Brooks, 2002).
“What type of vocational training or university degree did you receive?”
A handful of studies have found graduates of different fields to be differentially generous, although which groups are at the top is inconclusive (c.f., Bekkers & De Graaf, 2006; Belfield & Beney, 2000)
Not available for SOEP-IS
Income*
Higher income households donate higher amounts than lower ones, however, the relationship with discretionary income is complex and unresolved (McClelland & Brooks, 2004). Income elasticity has been shown to be a salient predictor (Brooks, 2005), but for our purposes, general net income seems like the most sensible since this is information EA organizations might be able to obtain or estimate.
“How satisfied are you with your household income?”
Unclear relationship: generally, appears to increase over time and level off around retirement, but this relationship is highly dependent on covariates such as church attendance, number of children, and marital status.
Should be available. I’m waiting for confirmation.
Number of children*
Positively related to philanthropy in most studies, but the age of the children may influence the direction and magnitude of the effect, specifically when they are younger than 14 (Okten & Osili, 2004) and 18 (Okunade & Berl, 1997).
According to ‘My Infratest’, these are the children in your household that were born in 2001 or later. Please state whether these children still live in your household.”
Mostly found to be positively related to giving, although a number of studies finding null effects (Apinunmahakul & Devlin, 2004; Carroll et al., 2006) call into question the magnitude of this effect.
The employed generally donate more than the unemployed (Chang, 2005a&b); those who work more (days and hours) donate more (Bekkers, 2004; Yamauchi & Yokoyama, 2005); retirees are highly charitable; self-employed are less generous (Carroll et al., 2006); and public service employees are more likely to engage in philanthropy than for-profit workers (Houston, 2006).
…could confirm officially unemployed: “Are you registered as unemployed at the Employment Office?”
Mixed findings in general and no finding when looking at one-person households (Andreoni et al., 2003). Still, given the ubiquity of this variable, it is sensible to include it in the model even though I have little faith it will be significant.
Should be available. I’m waiting for confirmation.
Race*
Caucasians generally give more, but this finding is tempered by the cause (non-whites donate more to the poor and religious organizations; Brooks, 2004; Brown & Ferris, 2007; Smith & Sikkink, 1998).
Should be available. I’m waiting for confirmation.
Parental background*
Higher levels of parental education, parental religious
involvement, and parental volunteering in the past are related to higher amounts currently donated by children (Bekkers 2005a). While current parental income and church attendance also predict giving (Lunn et al., 2001; Marr et al., 2005).
I thought a proxy for parent’s occupational prestige might be a salient predictor. Questions 496-502 cover the mother’s background and have the exact same wording.
Questions split depending on occupation and all contain the header: “What was your father’s occupational status as…”
Donations have been found to increase with emotional stability and extraversion (Bekkers, 2006b), as well as openness to experience (Levy et al., 2002). General social trust has also been found to be a salient predictor (Brooks, 2005; Micklewright & Schnepf, 2007). Empathy has been found to be related to donations (Bekkers & Wilhem, 2006), as well as altruism.
Big Five Personality traits:
Agreeableness: “is considerate and kind to others”
Persons with higher verbal scores (Bekkers & De Graaf, 2006), IQ (Millet & Dewitte, 2007), GPA (Marr et al., 2005), and ability to think in abstract terms (Levy et al., 2002) donate more.
Innovation exercise to assess emotional intelligence.
“What emotion was shown by the individual? For every emotion, please rate how strongly you perceived it. If you saw a group, please rate the emotion of the individual in the middle.”
For questions assessing quantitative skills (probabilities):
“Out of 1,000 people in a small town 500 are members of a choir. Out of these 500 members in the choir 100 are men. Out of the 500 inhabitants that are not in the choir 300 are men. What is the probability that a randomly drawn man is a member of the choir? Please indicate the probability in percent.”
Items 888-928 assess the ability to do expected utility calculations:
“Please imagine the following situation: You have the choice between a safe payment and a lottery. In detail: Do you prefer a 50% opportunity to win 300 Euro while you do not win anything by 50% or a safe payment of 160 Euro.”
“Now answer another question within 20 seconds. Continue the multiplication tables of the base 17 as far as possible. Starting with 17, 34, etc. The time is running - now.”
Donations are influenced by behavior of coworkers in the same salary quartile (positive; Carman, 2006), income inequality (negative; Okten & Osili, 2004), individualistic cultures (positive; Kemmelmeier et al., 2006), and the stock market (positive; Drezner, 2006).
Stock market optimism: “Initially we focus on the next year (next 12 months). Do you expect the DAX [German blue-chip index] to show rather profit or loss compared to the current value?”
Previously, no differences were found for secular donations (Brooks, 2005), but Fehr et al. (2019: 26) find that “for right-of-center respondents, there are indications that higher national relative income is related both correlationally and causally to more giving to poor Germans and Kenyans.”
Item designed by Fehr et al. (2019):
“In politics people often talk about ‘left’ and ‘right’ to mark different political attitudes. If you think about your own political attitude: Where would you place yourself?”
Persons with an internal locus of control are more likely to engage in philanthropy and other formal helping behaviors (Amato, 1985).
Ten item scale with the stem: “The following statements describe different attitudes towards life and the future. To which degree do you personally agree with the individual statements?”
Positive affect facilitates giving, while negative moods may also facilitate giving in specific circumstances but it is conditional on lots of factors (e.g., helping contains minimal barriers and when prompted to think about the negative feelings that would result from not helping; Cunningham et al., 1980; Weyant, 1978).
Short scale of emotions (angry, afraid, happy, sad):
“Thinking back on the past four weeks, please state how often you have experienced each of the following feelings very rarely, rarely, occasionally, often, or very often. How often have you felt...”
Endorse of prosocial values has a positive association with charitable giving. This is also true of individuals who are less materialistic (Sargeant et al., 2000) and care about justice (Todd & Lawson, 1999).
Questions 172-175 on justice. For example, the stem “To begin with it is about situations which result in others advantage and your disadvantage, because you were penalized, exploited or treated unfair. To what extent do you agree with the following statements?” Followed by “It makes me angry when other are undeservingly better off than me.”
Likelihood of events (e.g., financially successful, not get any serious illness, successful at work, content in general) happening compared to other people the same age and gender.
Spending money on others has been shown to have a consistent, causal impact on well-being (Aknin, Barrington-Leigh, Dunn, Helliwell, Biswas-Diener, Kemeza, Nyende, Ashton-James, & Norton, 2010). “One possibility is reverse causality, that is, that those who are inherently happier by nature are also more likely to help individuals” (Moynihan, DeLeire, & Enami, 2015).
“In conclusion, we would like to ask you about your satisfaction with your life in general. How satisfied are you with your life, all things considered?”
Cluelessness has been cited as a case against longtermism (Greaves & MacAskill, 2021). Thus, individuals that are predisposed to EA but are risk-adverse may be more likely to make global health and development donations.
Stem: “What do you think about yourself: How prepared to take risks are you in general?”
“not ready to take risk at all ... ready to take risk”
