Showing posts with label media. Show all posts
Showing posts with label media. Show all posts

Monday, December 28, 2009

Estimating black-white racial tension from 1850 to present

Previously I looked at how much attention elite whites have given to blacks since the 1870s by measuring the percent of all Harvard Crimson articles that contained the word "negro." That word stopped being used in any context after 1970, which doesn't allow us to see what's happened since then. Also, it is emotionally neutral, so while it tells us how much blacks were on the radar screen of whites, it doesn't suggest what emotions colored their conversations about race.

When tensions flare, people will start using the more charged words more frequently. The obvious counterpart to "negro" in this context is "nigger." It could be used by white racists, non-racists who are quoting or decrying white racists, by blacks trying to "re-claim" the term, by those debating whether or not the term should be used in any context, and so on. Basically, when racial tension is relatively low, these arguments don't come up as often, so the word won't appear as often.

I've searched the NYT back to 1852 and plotted how prevalent "nigger" was in a given year, though smoothing the data out using 5-year moving averages (click to enlarge):


We see high values leading up and throughout the Civil War, a comparatively lower level during Reconstruction, followed by two peaks that mark "the nadir of American race relations." It doesn't change much going through the 1920s, even though this is the period of the Great Migration of blacks from the South to the West and Northeast. It falls and stays pretty low during the worst part of the Great Depression, WWII, and the first 10 years after the war. This was a period of increasing racial consciousness and integration, and the prevalence of "negro" in the Crimson was increasing during this time as well. That means that there was a greater conversation taking place, but that it wasn't nasty in tone.

However, starting in the late 1950s it moves sharply upward, reaching a peak in 1971. This is the period of the Civil Rights movement, which on an objective level was merely continuing the previous trend of greater integration and dialogue. Yet just as we'd guess from what we've studied, the subjective quality of this phase of integration was much more acrimonious. Things start to calm down throughout the '70s and mid-'80s, which our study of history wouldn't lead us to suspect, but which a casual look at popular culture would support. Not only is this a period where pop music by blacks had little of a racial angle -- that was also true of most of the R&B music of most of the '60s -- but was explicitly about putting aside differences and moving on. This is most clearly shown in the disco music scene and its re-birth a few years later during the early '80s dance and pop music scene, when Rick James, Prince, and above all Michael Jackson tried to steer the culture onto a post-racial course.

But then the late '80s usher in a resurgence of identity politics based on race, sex, and sexual orientation ("political correctness," colloquially). The peak year here is technically 1995, but that is only because of the unusual weight given to the O.J. Simpson trial and Mark Fuhrman that year. Ignoring that, the real peak year of the racial tension was 1993 according to this measure. By the late '90s, the level has started to plummet, and the 2000s have been -- or should I say were -- relatively free of racial tension, a point I've made for awhile but that bears repeating since it's not commonly discussed.

Many people mention Obama's election, but that was pretty late in the stage. Think back to Hurricane Katrina and Kanye West trying but failing to foment another round of L.A. riots, or Al Sharpton trying but failing to turn the Jena Six into a civil rights cause celebre, or the mainstream media trying but failing to turn the Duke lacross hoax into a fact that would show how evil white people still are. We shouldn't be distracted by minor exceptions like right-thinking people casting out James Watson because that was an entirely elite and academic affair. It didn't set the entire country on fire. The same is true for the minor exception of Larry Summers being driven out of Harvard, which happened during a remarkably feminism-free time.

Indeed, it's hard to recognize the good times when they're happening -- unless they're fantastically good -- because losses loom larger than gains in our minds. Clearly racial tensions continue to go through cycles, no matter how much objective progress is made in improving the status of blacks relative to whites. Thus, we cannot expect further objective improvements to prevent another wave of racial tension.

Aside from the long mid-20th C hiatus, there are apparently 25 year distances between peaks, which is about one human generation. If the near future is like most of the past, we predict another peak around 2018, a prediction I've made before using similar reasoning about the length of time separating the general social hysterias that we've had -- although in those cases, just going back to perhaps the 1920s or 1900s, not all the way back to the 1850s. Still, right now we're in a fairly calm phase and we should enjoy it while it lasts. If you feel the urge to keep quiet on any sort of racial issues, you should err on the side of being more vocal for right now, since the mob isn't predicted to come out for another 5 years or so, and the peak not until 10 years from now. As a rough guide to which way the racial wind is blowing, simply ask yourself, "Does it feel like it did after Rodney King and the L.A. riots, or after the O.J. verdict?" If not, things aren't that bad.

Looking at absolute levels may be somewhat inaccurate -- maybe all that counts is where the upswings and downswings are. So I've also plotted the year-over-year percent change in how prevalent "nigger" is, though this time using 10-year moving averages to smooth the data out because yearly flucuations up or down are even more volatile than the underlying signal. In this graph, positive values mean the trend was moving upward, negative values mean it was moving downward, and values close to 0 mean it was staying fairly steady:


Again we see sustained positive growth during the Civil War, the two bookends of the nadir of race relations, although we now see a small amount of growth during the Harlem Renaissance era. The Civil Rights period jumps out the most. Here, the growth begins in the mid-1940s, but remember that it was at its lowest absolute levels then, so even the modest increases that began then show up as large percent increases. The PC era of the late '80s through the mid '90s also clearly shows up. There are several periods of relative stasis, but I see three periods of decisively moving against a nasty and bitter tone in our racial conversations: Reconstruction after the Civil War (admittedly not very long or very deep), the late '30s through WWII, and the "these are the good times" / Prince / Michael Jackson era of the mid-late '70s through the mid '80s, which is the most pronounced of all.

That trend also showed up television, when black-oriented sitcoms were incredibly popular. During the 1974-'75 season, 3 of the top 10 TV shows were Good Times, Sanford and Son, and The Jeffersons. The last of those that were national hits, at least as far as I recall, were The Cosby Show, A Different World, Family Matters, The Fresh Prince of Bel-Air, and In Living Color, which were most popular in the late '80s and early '90s. Diff'rent Strokes spans this period perfectly in theme and in time, featuring an integrated cast (and not in the form of a "token black guy") and lasting from 1978 to 1986. The PC movement and its aftermath pretty much killed off the widely appealing black sitcom, although after a quick search, I see that Disney had a top-rated show called That's So Raven in the middle of the tension-free 2000s. But it's hard to think of black-focused shows from the mid-'90s through the early 2000s that were as popular as Good Times or The Cosby Show.

But enough about TV. The point is simply that the academic material we're taught in school usually doesn't take into account what's popular on the radio or TV -- the people's culture only counts if they wrote songs about walking the picket line, showed that women too can be mechanics, or that we shall overcome. Historians, and people generally, are biased to see things as bad and getting worse, so they rarely notice when things were pretty good. But some aspects of popular culture can shed light on what was really going on because its producers are not academics with an axe to grind but entrepreneurs who need to know their audience and stay in touch with the times.

Tuesday, December 15, 2009

Brief: Is the "culture of fear" irrational?

We hear a lot about how paranoid Americans are about certain things -- people in the middle of nowhere fearing that they could be the next target of a terrorist attack, consumers suspicious of everything they eat because they heard a news story about it causing cancer, and so on.

Of course, we could be overreacting to the magnitude of the problem, as when we panic about a scenario that has a 1 in a trillion chance of occurring but that sounds disastrous if it did happen. It's not clear, though, what the "appropriate" level of concern should be for a disaster of a given magnitude and chance of happening. So the charge of irrationality is harder to level using this argument about a single event.

But we also get comparisons of risk between events wrong, for example when we fear traveling by airplane more than traveling by car, even though planes are safer. Here the case for irrationality is straightforward: for a given level of disaster (say, breaking your arm, dying, or whatever), we should panic more about the more probable ways that it can occur. The plane vs. car example makes us look irrational.

Still, there's another way we could measure how sensible our response is, only instead of comparing two sources of danger at the same point in time, comparing the same source of danger at different points in time. That is, for a given level of disaster, any change in the probability of it happening over time should cause us to adjust our level of concern accordingly. If dying in a plane crash becomes less and less likely over time, people should become less and less afraid of flying. When I looked into this before, I found that the NYT's coverage of murder and rape has become increasing out-of-touch with reality: while the crime statistics show the murder and rape rates falling after the early 1990s, the NYT devoted more and more of its articles to these crimes. So at least at the Newspaper of Record, they were responding irrationally to danger.

But what about the average American -- maybe the NYT responds in the opposite way that we might expect because when violent crime is high, people see and hear about plenty of awful things outside of the media, so that writing tons of articles about murder and rape wouldn't draw in lots more readers. In contrast, when the society becomes safer and safer, an article about murder or rape is suddenly shocking -- just when you thought things were safe! -- and so draws more readers, who start to doubt their declining concern about violence.

The General Social Survey asks people whether there's any area within a mile of their house where they're afraid to go out at night. Here is a plot of the percent of people who say that they are afraid to go out at night, along with the homicide rate for that year:


Clearly there is a tight fit between people's perception of danger and the reality underlying that fear. The Spearman rank correlation between the two within a given year is +0.74. That's assuming that people respond very quickly to changes in violence; it might be even higher because there appears to be somewhat of a lag between a change in the homicide rate and an appropriate change in the level of fear. For example, the homicide rate starts to decline steadily after a peak in 1991, but people's fear doesn't peak until two years later, when it too steadily declines. That makes sense: even if you read the crime statistics, those don't come out until two years later. To respond right away, you'd have to be involved in the collection and analysis of those data. The delay is more likely due to people hearing through word-of-mouth that things are getting better -- or not getting negative word-of-mouth reports -- and that it takes awhile for this information to spread throughout people's social network.

Putting all of the data together presents a mixed picture on how rational or irrational our response to the risk of danger is. But here is one solid piece that average people -- not those with an incentive to misrepresent reality, either in a more negative or more positive way -- do respond rationally to risk.

GSS variables used: fear, year

Monday, December 14, 2009

Brief: Which Western countries care most about preserving the media?

A recent NYT article reviews a German study of how willing people in various countries are to pay for "online content" -- news reporting, videos, songs, etc. In the German-language PDF linked to above, a table shows how people in the countries studied describe their preferences for getting online content. The columns read: free with ads, free with no ads, pay with no ads, pay with ads, and none of above. You can probably figure out what the country names are.

There's a fair amount of variation in how much of the population is willing to pay at all, or conditional on paying or not paying, whether they'll accept advertising. To see what explains this, I've excluded people who answered "none of the above," leaving only those who expressed an opinion. In the table below, I've lumped the two "free" groups together and the two "pay" groups together. I've also calculated a "delusional" index, which answers the question, "Of those who want free content, what percent expect it to not even have advertising?" Obviously someone has to pay for news articles to get written, and some free-preferers recognize that advertising is the only viable alternative to not paying for typical media products yourself. So, those who answer "free, no ads" are expecting media producers to behave like charities.

The table also shows GDP (PPP) per capita, on the hunch that richer countries will be more willing to pay for news, videos, and so on. In economics jargon, this online content is a "normal good" that sees a rise in demand when a population gets wealthier (an "income effect" pushes the demand curve up). The table is ordered from most to least willing to pay.

Country GDP / cap % Pay Delusional
Sweden 37,334 25 20
Netherlands 40,558 20 36
Great Britain 36,358 19 43
USA 47,440 18 27
Belgium 36,416 15 33
Italy 30,631 15 46
Greece 30,681 13 27
Bulgaria 12,322 12 41
Czech 25,118 11 56
Germany 35,539 10 56
Turkey 13,139 9 54
Romania 12,600 9 48
France 34,205 8 56
Portugal 22,232 8 52
Hungary 19,553 8 52
Spain 30,589 6 61
Poland 17,537 5 57


It sure looks like wealth plays a key role, so let's look at how GDP relates to both willingness to pay and how delusional the free-preferers are:


Since the willingness to pay and the delusional index cannot vary outside of the range [0,1], I use the Spearman rank correlation instead of the Pearson correlation. The correlation between GDP and the percent willing to pay is +0.67 (two-tailed p less than 0.01). Between GDP and the delusional index, it is -0.49 (two-tailed p = 0.05). People in richer countries are more willing to pay, and they are less deluded about where stuff comes from -- that is, typically not from charities.

The surprises are worth looking at. At the top, we see mostly Anglo and Nordic countries, and at the bottom the more southern and eastern parts of Europe. However, France ranks pretty low, and Germany doesn't do much better, even though we Americans think of those countries as having more sophisticated media tastes, and as treasuring the institutions of the media -- Gutenberg in Germany, for example. We also think of ourselves as much more bratty when it comes to the media -- that because our sophistication level is so low, we're only a tiny bit willing to pay, and any more than that we'll junk the news in favor of some other cheap form of entertainment.

It looks like the ones that have more pro-market views are more supportive of paying for online content. The French and Germans may like the idea of keeping the media alive and thriving, but Americans are more willing to do what it takes to ensure that happens.

Monday, August 31, 2009

Youthful exuberance: Age and the housing bubble

The internet does not produce any non-self-referential data; it simply makes the data easier to find and faster to obtain. Still, if this drops below some threshold, some people might get into investigation who might not otherwise would have bothered -- and that's as good as producing new data, since unexamined data do not really exist. Probably the best result of this is the ability of people with computers to figure out if a popular story is true or not.

One story that was everywhere in the media not too long ago -- although it has probably since been crowded out by all the news about the collapse of the global economy -- was that young people today refuse to grow up. In particular, they were supposed to be opting more than ever before to live at home and freeload off their parents well into their 30s. This story broke into the mainstream when Time Magazine ran a cover story about twixters, the goofy name for these aging slackers. Around the same time, MTV devoted an episode of True Life (a popular hour-long documentary show with different topics each week) to young people who were moving back with their parents.

These stories were circulating during 2005, at the height of the real estate euphoria, so the inference is that young people were being left out of the housing bubble. The greatest beneficiaries must have been older people starting a family, or perhaps middle-aged speculators looking to "flip that house." But rather than take the tales at face value, let's see how well or how badly the bubble treated young people, using housing data from the Statistical Abstract of the United States, which begin in 1982 and go through 2007.

First, here is the distribution of homeowners by age, using coarser and finer-grained age groups:





It would seem from the first chart that homeowners were becoming grayer, but the youngest category is pretty wide -- "under 35." Looking at the finer-grained chart, we see that those under 30 seemed to be doing pretty well, especially since their low in the early 1990s. This looks especially true for the under-25 group, but it's hard to see in the chart. To get a better view, here is the homeownership rate for the under-25 age group over time:


Indeed, they were treated quite well by housing bubble that began in the late 1990s. Just eye-balling it, there is a gain of about 10 percentage points -- or about a 67% increase. But maybe they were nothing special, and the other age groups had similar increases. Let's have a look at how their rates changed from the early-'90s low to the peak in the mid-2000s:


The middle-aged age groups and above tend to have homeownership rates of at least 70%, often 80% or higher, so there is only so much room for them to gain before they hit 100%. That's why their increases aren't so great. The increases among the elderly reflect a steady increase since 1982, unlike the ups and downs that every other age group went through, so they're a different story.

But for the younger age groups, all of them had lots of room to gain, and yet it was still the very youngest whose rates shot up the most in relative terms. Clearly we were all fed a bunch of bull about 20-somethings refusing to make it on their own, mooching off their parents forever, and so on. A priori, that story should have sounded implausible since the 20-somethings of 2005 had come of age during two frenetic economic booms separated by a quite tepid recession. It sounds instead like the world was their oyster.

(I was a mid-20-something living and working in Barcelona at the time-- something I couldn't dream of doing now, with the Spanish economy in ruins. And even though housing became cheap after their real estate bubble burst, it wouldn't be any fun this time around since the euphoria has dissipated.)

How can we get a good intuitive feel for how spectacular the housing bubble was for young people? Let's compare their percent increase in homeownership rates to other not-so-credit-worthy groups. Many commentators have pointed to the increase of homeownership among Blacks and Hispanics, but their rates only increased by just over 10% and 20%, respectively. They started fairly low too -- not like the middle-aged -- so their increase was similar to the 25 - 29 and 30 - 34 age groups. Nothing close to the gains of the under-25 group. The same is true for single mothers, whose rates increased by under 20%.

As far as I can tell, young people were the greatest beneficiaries of the drive to debauch lending standards, especially the move to do away with down payments. After all, how much money could you have saved up by that age to put down? Forget the fact that what little you did earn you probably piddled away on your car, electronics, beer, shoes, handbags, and so on.

These data underscore the importance of paying attention to age as a demographic variable, something that we rarely do, except to whine about how youth-obsessed our culture is (or is becoming). That's not true either, but in any case, we already have lots of analysis based on race or ethnicity, sex, class, and even sexual orientation. Age is typically a far stronger cause of differences than any of those, yet we neglect it too often. This may reflect how homogeneous our social circles are by age, so that differences across age groups don't automatically spring to mind. Still, we should make a conscious effort to pay better attention to age when trying to figure out how people and their societies work.

Saturday, August 29, 2009

Brief: Have we gotten more or less sympathetic since Adam Smith's time?

In his Theory of Moral Sentiments, Adam Smith made the observation that we care less about the disasters that befall others if they are remote and faceless, while we panic at much smaller hardships of our own. But he wrote that before the Industrial Revolution really took off, and so before the peace-and-life-valuing merchant classes genetically replaced the old warring aristocratic class. In the meantime, capital punishment has been widely banned across Europe and its off-shoots, we have laws against cruelty to animals, and we have TV and other media that provide us with vivid daily images of the troubles that beset people in far-off places. So, does his observation still hold up in our more bleeding heart times?

To check this, I searched the NYT for its coverage of a rival first-world country, Japan, and a non-threatening third-world country, Indonesia. If newspapers cover a country out of sympathy for their plight -- and this supply would reflect demand for such coverage -- then there should be declining coverage of Japan during their incredible boom of the 1980s, but increasingly more coverage as they slid into the Lost Decade of the 1990s and even early half of the 2000s. Similarly, coverage of Indonesia should spike during 2004 - 2005 in the wake of the disastrous tsunami, which hit Indonesia much harder than other countries. (This event was pretty close to Smith's hypothetical earthquake that swallowed all of China.) There should also be a spike during their financial crisis of 1997 - 1998, although this was not a deadly catastrophe that could provide the level of gory detail as a tsunami, so this spike should be smaller than that of the tsunami coverage.

Here is the NYT's coverage of these countries:




Counter to the sympathy hypothesis, coverage of Japan shot up during the 1980s, as Americans began to fear more and more that Japan was going to economically take over our country. When Japan slid into a long recession, those fears evaporated, and the supply of alarming stories declined as a result, all the way to the present. This is not to say that there was no demand for sympathy stories about the Japanese recession, only that fear is a stronger driver of coverage than sympathy.

The Indonesia data give some support to the sympathy hypothesis. There is indeed a two-year increase for 2004 - 2005, reversing a previous steady decline. After that moment of sympathy, though, coverage starts to decline again. Moreover, the upward blip during the tsunami is tiny compared to the skyrocketing coverage of 1997 - 1998, when Indonesia was rocked by a financial crisis. This reflects our panic that the Asian Financial Crisis would infect our own economy. Before this threat to us, we paid relatively little -- and consistently little -- attention to Indonesia.

On the whole, the coverage data support Adam Smith's claim, despite our being more emotionally sensitive than people were during his time. However, there is no real paradox here if we view the demand for sympathy stories -- and the coverage that supplies that demand -- as a function of both some baseline concern for others plus a component that responds to actual disasters. Let's say that sympathy is a simple, linear function of disasters:

S = b + r*D

Where S is level of sympathy, b is our baseline level in the absence of news about disasters, D is the level of disasters that we hear about, and r measures how responsive our sympathy is to those disasters.

All that Smith was saying is that r is greater for disasters that are nearer to us than for disasters that are more remote. He made no claim about our baseline level. The genetic and cultural proliferation of the merchant classes -- and the concomitant doing away with public executions, slavery, etc. -- may have increased the baseline without affecting how r responds to disasters at different social distances. This seems like a useful distinction to draw, as it clears up a lot of the confusion about whether we've become more selfish or sensitive in recent centuries.