|
Jim Thomas, Professor of Sociology and Criminal Justice at
Northern Illinois University, made the following criticisms
of the Time article and the Rimm study in the WELL Media Conference,
topic 1029.
jthomas@well.com
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
media.1029.58: jim thomas (jthomas) Sun 25 Jun 95 21:09
From Rimm's comment to Mike:
> Our conclusions are very clear: there are no
> differences when communities are defined by telephone area codes. This
> there are no ~community standards~on which communities differ.
They are in no position to make this claim unless they also have
demographic and preference data on individual communities. One cannot
infer "community standards" from download data that, in all probability,
does not represent the erotica-using segment of a given population.
Nor can" be assessed by erotica preferences.
This snippet reflects either unprofessional sloppy wording or
methodological incompetence. Thanks to Donna for pointing out that
Rimm is an undergrad. My first feeling when I read abstract was it resembled undergrad writing methods classes,
professional research. recall receiving some material long while back regarding project.'ll try dig archives (if find it). put off then simplicity, and nothing've here changes mind. abstract:
> As Americans become increasingly computer literate, they are discovering
> an unusual and exploding repertoire of sexually explicit imagery on the
> Usenet and on "adult" computer bulletin board services (BBS). Every time
> they log on, their transactions assist pornographers in compiling
> databases of information about their buying habits and sexual tastes.
The conflation of Usenet and BBSes here is either dramatic hyperbole
or a fatal methodological flaw. It cannot imagine it passing muster
in a reputable social science peer-reviewed journal.
The abstract conflates erotica and pornography. Let's hope that the
study pays somewhat closer attention to operational variables.
Donna writes with her usual acumen:
> In my entire professional career, I have *never* heard of a scholarly
> journal embargoing a study until date of publication to protect the authors'
> "intellectual property." In any event, in most fields this would be
> impossible because the working papers and revisions are freely circulated
> for months and sometimes years ahead of publication.
Ditto. Compare this with the Laumann (et. al) U of Chicago sex survey.
Conference papers, peer review, revisions, and sharpening of the link
between data and interpretation are part of the process. It saddens me
that ped did not recognize this in his TIME story. I'm astonished that
he gives both equal credibility by focusing on the "findings" and not
the method. Readers of the discussion of SEX IN AMERICA here on the Well
will recall the intense methodological critiques. Why, ped, did you
give the RIMM study, done by undergrads, a methodological pass?
I'll withhold further judgment until I see the actual study. It's
likely that the authors have some interesting data. But, judging from
what I've read above, there is no indication that the authors have an
adequate grasp of acceptable scientific method and how to interpret
that data.
media.1029.61: jim thomas (jthomas) Sun 25 Jun 95 21:31
I've just finished the Time stories on AOL. Josh Quittner's piece on
developing software that will allow parents to block access to
unacceptable on-line material is quite good. Wendy Cole's profile of
the AA BBS case is also nicely done. The cover story is nothing to
get upset about. In fact, it's simply trite. It's not up to ped's usual
high quality, and as a fan of his, I was disappointed that he did not
use the opportunity to delve into the complexity of the
issues. Instead, he simply rehashed the same old stuff--sex is popular,
it's on computers, and it's controversial. Nothing new here. What I
find most disturbing is his uncritical use of the Rimm study as a hook,
a kind of teaser, to anchor his discussion.
[Note: The above was written before I'd seen the cover art, read the
study itself, or read subsequent information about the study and
Philip's responses]
Call me old fashioned, but I'd rather not see reporters give the same
credibility to an unpublished, non-peer reviewed, "secret" study
by undergrads as, say, the Laumann or even the Bell Curve were given,
until that study can be independently evaluated.
But, then, I'm not a journalist looking for a sexy cover story.
media.1029.78: jim thomas (jthomas) Mon 26 Jun 95 10:58
> Maybe you should read it again, a litte more carefully, Jon. The first
> several paragraphs discuss the pervasiveness of sex *outside* the net, not
> the Rimm report.
Uh, ped, you might want to re-read the first sentence of the story
you wrote:
"It's popular, pervasive and surprisingly perverse, according to the first
^^^^^^^^^^^^^^^^^^^^^^
survey of online erotica. And there's no easy way to stamp it out."
^^^^^^^^^^^^^^^^^^^^^^^^^
Reads to me like you're trying to hook the reader with a juicy teaser
with the Rimm study as the hook. Your second paragraph certainly pulled
me in with your talk of "balloon breasted models" and all those
"unflagging erections." Your third para brings it back to computers.
I think Jon is accurate.
Your first sentence, third para, would have made a nifty lead:
>Something about the combination of sex and computers, however,
>seems to make otherwise worldly-wise adults a little crazy. How else
>to explain the uproar........(ect)
From there, the story writes itself, and you could have alluded to the
Rimm study in a more appropriate context. That would have made a much
better story than the one promised by online erotica, Rimm, and erections.
media.1029.90: jim thomas (jthomas) Mon 26 Jun 95 16:39
I've just read the Newsweek piece "No Place for Kids? A Parent's Guide to
Sex on the Net" by Steven Levy. The piece looks as if it were written
by committee--the intro is quite good (so it must be Steven's):
"When the annals of cyberspace are uploaded for future generations,
digital historians will undoubtedly include a scene from the Senate
chamber erlier this month: Nebraska Democrat James Exon brandishing
a thin binder now known as the blue book. Inside were images snatched
from the shadows and thrust into the center of public discourse.
Women bound and being burned by cigarettes. Pierces with swords.
Having sex with a German sheperd. As Exon puts it, images that
are "repulsive and far off bas." Images from the Net."
Who needs the Net when we have Newsweek and Time to stimulate our
fantasies? Weren't those images from private, adult-only BBSes (actually,
a single BBS)?
The contents (p 3) lays out the motif nicely:
"A PARENTS GUIDE TO NET SMUT
Full-color porn, predators who use e-mail
to lure kids, alt.sex newsgroups: cyberspace seems like a cesspoool.
How to protect kids against the Net's carnal pitfalls. Technology: p 46.
((graphic on the left of a chesty babe that appears to have be an R-rated
GIF-type file photographed on a MAC viewer. Caption:))
In cyberspace someone's always half-dressed.
Rimm's study "provides solid evidence that there's loads of hard-core stuff
in cyberspace.......his conclusion: 'I think there's almost no
question that we're seeing an unprecedented availability and
demand of material like sadomasochism, bestiality, vaginal and rectal
fisting, eroticized urinatin...and pedophilia."
Accompanying pictures of "sleaze" give the mistaken impression that erotica
hits users in the face at every turn.
media.1029.135: jim thomas (jthomas) Mon 26 Jun 95 22:21
ped writes:
Jon028 writes:
>> Uh, ped, you might want to re-read the first sentence of the story
>> you wrote:
>>> "It's popular, pervasive and surprisingly perverse, according to the first
^^^^^^^^^^^^^^^^^^^^^
>>> survey of online erotica. And there's no easy way to stamp it out."
>>> You probably had now way of knowing from the ascii version, Jon, but that's
>>> a headline, not the first sentence. Understandable confusion.
No, that was me, jthomas, who wrote that.
There was no confusion. Perhaps, like the Ashanti, when our god-priests
reveal their baser mortality, we are more inclined to devour them. As
one of your long-time admirers, I'm saddened to see your ploys. I'm
also wondering if you read what you write, because the only confusion
here seems to be yours. Here is what you've described as the headline,
followed by your opening paragraphs:
>It's popular, pervasive and surprisingly perverse, according to the first
>survey of online erotica. And there's no easy way to stamp it out.
The headline? Fine. You claimed that your first two paragraphs dealt with general pervasiveness, not the Net.
Here are your first several paragraphs:
--- begin ped story ---
Sex
is everywhere these days--in books, magazines, films, television, music
videos and bus-stop perfume ads. It is printed on dial-a-porn business
cards and slipped under windshield wipers. It is acted out by balloon-breasted
models and actors with unflagging erections, then rented for $4 a night
at the corner video store. Most Americans have become so inured to the
open display of eroticism--and the arguments for why it enjoys special
status under the First Amendment--that they hardly notice it's there.
Something about the combination of sex and computers, however, seems
to make otherwise worldly-wise adults a little crazy. How else to explain
the uproar surrounding the discovery by a U.S. Senator--Nebraska Democrat
James Exon--that pornographic pictures can be downloaded from the Internet
and displayed on a home computer? This, as any computer-savvy undergrad
can testify, is old news. Yet suddenly the press is on alert, parents
and teachers are up in arms, and lawmakers in Washington are rushing
to ban the smut from cyberspace with new legislation--sometimes with
little regard to either its effectiveness or its constitutionality.
If you think things are crazy now, though, wait until the politicians
get hold of a report coming out this week. A research team at Carnegie
Mellon University in Pittsburgh, Pennsylvania, has conducted an exhaustive
study of online porn--what's available, who is downloading it, what
turns them on--and the findings (to be published in the Georgetown Law
Journal) are sure to pour fuel on an already explosive debate.
The study,
titled Marketing Pornography on the Information Superhighway, is significant
not only for what it tells us about what's happening on the computer
networks but also for what it tells us about ourselves. Pornography's
appeal is surprisingly elusive. It plays as much on fear, anxiety, curiosity
and taboo as on genuine eroticism. The Carnegie Mellon study, drawing
on elaborate computer records of online activity, was able to measure
for the first time what people actually download, rather than what they
say they want to see. "We now know what the consumers of computer pornography
really look at in the privacy of their own homes," says Marty Rimm,
the study's principal investigator. "And we're finding a fundamental
shift in the kinds of images they demand."
--- end first four paras of ped story ---
Your first paragraph does talk about general pervasiveness
of sex, without mentioning the Internet. So what? It's the context by
which you titillate your audience with cocks and tits for the Net-tie
in the second paragraph. Your second paragraph is itself innocuous,
and from it, you could take your story in several directions, including
the press bandwagon affect to which you allude. Those first paragraphs
set up the lead-in to the Rimm study, which you call "exhaustive" and
"significant." You call it a "Carnegie Mellon study," which gives it
a credibility it may not have. In fact, judging from what I've read
here, it's a study by some untrained undergraduates who received research
funds for the study, funds that in most schools are available to many
students and faculty.
Given the play you gave the study and the importance
of it to your article, what measures did you take to assure that it
was a sound study before using it as the basis for many of your exaggerated
claims and central hook? It's one thing to allude to a study--even a
poor one. It's quite another to praise it as you did without providing
the justification. Think back to The Bell Curve and how different the
treatment of the methodology was.
Now, please explain again how your
opening paragraphs deal with the general pervasiveness of sex rather
than to the Internet link?
media.1029.146: jim thomas (jthomas) Mon 26 Jun 95 22:48 scribbled by jthomas Mon 26 Jun 95 23:12
media.1029.155:
jim thomas (jthomas) Mon 26 Jun 95 23:22
I scribbled 146, which was
a response to Steven. I've re-read his piece for the third time, and
while I have some quibbles with his reporting of the Rimm study and
a few other minor points, the story itself isn't bad. I retract my "written
by committee" judgment, which was influenced by the story's graphics
and captions.
Steven doesn't sensationalize sex. Instead, he focuses
on the legimate issues of how legislators, parents, and others, have
responded to the sex hysteria.
My guess is that his message will be
lost in the hype that the graphics, captions, and teasers dramatize.
media.1029.158: jim thomas (jthomas) Mon 26 Jun 95 23:39
Both Time and
Newsweek alluded to the 917,410 figure. Newsweek indicated that the
figure alluded to the picture's descriptions. The Time story said:
>In an 18-month study, the team surveyed 917,410 sexually explicit
>pictures, descriptions, short stories and film clips. On those
>Usenet newsgroups where digitized images are stored, 83.5
>percent of the pictures were pornographic.
"Surveyed 917,410?" What does this mean? Normally, one
surveys people, not pictures. Were they "sampled?" The Time story explicitly
conveys the message that SEXUALLY EXPLICIT PICTURES, DESCRIPTIONS,
SHORT STORIES, and FILM CLIPS were given individual attention. Now,
how does an AI program process a film clip?
ped, didn't a bell go off?
Let's assume that ONLY descriptions were processed. And, let's assume
that it took only a total of two minutes to download each description
and get it in a form where it could be processed by AI software. That's:
30,580 hours
1,274 days
Assuming four researchers working 24 hours a
day, that's: 318 days each Kinda impressive for 18 months (548 days)
work. And that doesn't count calling time, writing stuff up, or other
minor stuff.
ped, maybe your figures are accurate, but you've sure left
out some rather important information that even an undergrad social
science student is trained to ask. And, if the figures are, in fact,
accurate, then the story isn't Net-sex, but mega-endurance.
media.1029.161: jim thomas (jthomas) Tue 27 Jun 95 00:42
>If you think things are crazy now, though, wait until the politicians get
>hold of a report coming out this week.
Wait until they get hold of the Time article!
>"We now know what the consumers of computer
>pornography really look at in the privacy of their own homes," says Marty
>Rimm, the study's principal investigator. "And we're finding a fundamental
>shift in the kinds of mages they demand."
Did Rimm also provide data on non-Net erotica consumption
to allow him, and you, to make this claim? Or is this just another possible
methodological stretch that you ignored?
>On those Usenet newsgroups where digitized images are stored, 83.5
>percent of the pictures were pornographic.
Perhaps it's just ped's sloppy wording, or perhaps it's
simply uncritical repitition of a line in the Rimm study. That sentence
conveys the impression that in any Usenet group, over 80 percent of
the pictures will be "pornographic." Imagine, ped, the response to your
story of a mother who's child hangs out on astronomy group where digitized
images are available. Are you really telling her that four out of five
pictures there will be "pornographic?"
How did the study operationalize
"pornography" for the purposes of classification? Would a Playboy centerfold
be considered pornographic? What about a wet t-shirt model? ped, given
the implications of such a story, did it ever occur to you ask or report
definitions? Compare this story with the treatment given Bell Curve.
Can you tell the difference?
>IT IS UBIQUITOUS. Using data obtainedwith permission from BBS operators, the
>Carnegie Mellon team identified (but did not publish the names of) individual
>consumers in more than 2,000 cities in all 50 states and 40 countries,
>territories and provinces around the world--including some countries like
>China, where possession of pornography can be a capital offense.
Perhaps Rimm, et. al., did
convince BBS operators to provide the names of individual consumers.
On some boards, users are listed in a Users' file, and such files are
hardly accurate. ped, did you ask Rimm any critical questions at all?
Didn't tidbits like this, coupled with the fact that the study was neither
peer reviewed nor public, pique your investigative curiosity?
>IT IS A GUY THING. According to the BBS operators, 98.9 percent of the
>consumers of online porn are men. And there is some evidence that many of the
>remaining 1.1 percent are women paid to hang out on the "chat" rooms and
>bulletin boards to make the patrons feel more comfortable.
ped,
how would BBS operators know how many online consumers are male? Or,
are you eliding BBSes and the Net here? What evidence is there that
"many" of that remaining 1.1 percent are women paid to hang out online?
Please cite a reliable source for your evidence, ped. Otherwise you
should retract it. Did that tidbit come from the study? If so, what
was the evidence?
>And as the Carnegie Mellon
>study is careful to point out, pornographic image files, despite their
>evident popularity, represent only about 3 percent of all the messages on the
>Usenet newsgroups, while the Usenet itself represents only 11.5 percent of
>the traffic on the Internet.
ped, please provide the precise statistic(s) for this
claim. The Usenet arbitron figures in the past do not support this figure
of roughly one in 30 Usenet messages being a "pornogaphic image file."
Think about that figure for a minute. Even if we assumed that one image
was broken into ten separate files, it still wouldn't come to the figure
you gave. Or, does that three perecent refer to file volume? ped, you
have just reported to millions of your readers a figure that seems wildly
inaccurate. Now, perhaps that figure is correct. I ask only: a) did
you check it? b) if so, can you verify it? And, why did you not mention
also that Internet traffic accounts for only a portion of all Net traffic?
ped, how did the investigators tabulate alt.sex* groups? Did they assume
that all posts to alt.sex.fuckme.fuckme.fuckme were, by definition "pornographic?"
Or did they (or their AI program) distinguish between erotic, non-erotic,
and posts that simply said "fuck you?" Is this explained in the study's
methodology? If not, did you bother to ask?
To me, it looks as if you
are uncritically accepting figures that play to the Net-sex hysteria.
I called your story itself no big deal. By that I meant it's trite and
uninformed. That does not mean that I think the consequences of your
story are no big deal. You appear to have been sucked into the "porn
hysteria" and have succeeded in reducing public understanding the lowest
possible denominator of ignorance and hyperbole.
media.1029.162: jim thomas (jthomas) Tue 27 Jun 95 00:55
> Mike, thse are two separate sentences and two separate statements. They
> surveyed nearly a million pictures from adult BBSs.
ped, that was a two sentence paragraph. Only the most
disingenuous waffling can disclaim as you do the connection between
the two sentences.
Why not just admit what is indisputable? Either your
facts or your wording were sloppy. And, whether they actually "surveyed"
nearly one million pictures, stories, descriptions, and the rest, remains
disputable. In your story, you claimed that they "surveyed" a broad
range of material. Above, you limit it to pictures. Which is it?
media.1029.186: jim thomas (jthomas) Tue 27 Jun 95 12:20
ped writes: >When I posted above that I hadn't
>heard any criticism of the study's methodology beyond the fact
that it was
>conducted by an undergraduate, I meant I had hadn't heard any *here."
I had,
^^^^^^
>of course, heard Mike and prof criticize it. I also had reporting from three
>of the Sex in America researchers. The thrust of what they had to say got
>into the story.
??????
- Despite the fact that you won't share the study, there
in fact have been ample concerns raised about potential methodological
problems. I'll be happy to dig them out of the posts prior to your
original #95 and since.
- You adduce the SiA researchers to bolster
the methodological integrity of the study. Did Laumann et. al. read
the study (as you strongly imply)? Or, did you (or somebody else?)
simply ask for some quick quotes.
The "thrust of what they had to say got into the story," ped?
Here's what you wrote:
--- begin two paragraphs ---
How accurately these images reflect America's sexual
interests, however, is a matter of some dispute. University of Chicago
sociologist Edward Laumann, whose 1994 Sex in America survey painted
a far more humdrum picture of America's sex life, says the Carnegie
Mellon study may have captured what he calls the "gaper phenomenon."
"There is a curiosity for things that are extraordinary and way
out," he says. "It's like driving by a horrible accident. No one
wants to be in it, but we all slow down to watch."
Other sociologists
point out that the difference between the Chicago and Carnegie Mellon
reports may be more apparent than real. Those 1 million or 2 million
people who download pictures from the Internet represent a self-selected
group with an interest in erotica. The Sex in America respondents,
by contrast, were a few thousand people selected to represent a
cross section of all America. Still, the new research is a gold
mine for psychologists, social scientists, computer marketers and
anybody with an interest in human sexual behavior.
--- end two paragraphs ---
Re-read your response to Mike, ped. You clearly adduce Laumann
to support your acceptance of the methodology. Now, please re-read
your Laumann quote. Not only does it sound like they had not read
the study, but they say virtually nothing about method. In fact,
the subtext of what you report is more easily interpreted as a back-handed
swipe at the study.
I'll leave it to other methodologists to count
the non sequitors in ped's second paragraph above. Normally, they
could be overlooked as simple lack of familiarization with a topic.
However, given ped's refusal to acknowledge fundamental errors,
they take on new significance. ped might also reflect on how that
last sentence in the second para directly imbues the study additional
credibility that he remains unable to defend.
I was not particularly
angry with ped for the story. But, I am finding his disingenuousness
in replies and his lack of recall on what he, himself, wrote in
the story, to be highly distressing.
media.1029.188: jim thomas jthomas) Tue 27 Jun 95 12:36
Re comments that the Laumann study took methodological hits:
Even superficial readers of the exchanges
between some of the methodologists here will recall the debates
as intense, specific, and focusing on the minute details of their
sampling, survey procedure, and interpretive techniques. This was
done in the same way that any reputable research is critiqued, which
cannot be said for the Rimm report. Some of the criticisms were
convincing, some were not. That a study has methodological problems
does not mean that it therefore lacks value. Rather, the critiques
provide some guidance for the readers in assessing the credibility
of claims and in interpreting results.
Because none of us have seen
the Rimm methodology, we can only rely on what was reported in the
abstract, the article(s) and a few posts above. Even from this limited
information, there are serious questions that arise about the conceptualization,
logic, data-gathering, and interpretive procedures.
Much of this
could be put to rest if ped would provide some of us with a copy
of the study.
media.1029.199: jim thomas (jthomas) Tue 27 Jun 95 16:06
> If ped has a lack of recollection for stories he wrote, I can't blame him; I
> work for a weekly and I know how it is. :-)
No disagreement in principle, Sharon. I can rarely remember the
titles of what I wrote.
The difference here is that the words have
been reproduced in this forum. Criticizing others for misreading,
or making errors that could be avoided by re-reading the reproduced
snippets.
This all must be terribly difficult for ped, who's trying
to enjoy some downtime with his family. Although my concerns are
unabated, ped deserves credit for hanging in. I remain distressed
with some of his commentary, but I respect how he's tried to address
the questions without rancor.
media.1029.247: jim thomas (jthomas) Tue 27 Jun 95 22:01
Here's the text that pops up at http://TRFN.pgh.pa.us/guest/mrstudy.html
The first paragraph is identical with ped's depiction. It should
have set bells jangling. Note the cryptic allusion to "conspiracies"
in the last para of page 2:
-----------------------
Guest Page: Marketing Pornography on the Information Superhighway (p1 of 2)
"MARKETING PORNOGRAPHY ON THE INFORMATION SUPERHIGHWAY: A Survey
of 917,410 Images, Description, Short Stories and Animations Downloaded
8.5 Million Times by Consumers in Over 2000 Cities in Forty Countries,
Provinces and Territories"
Disclaimer: The Three Rivers FreeNet,
and its related entities, have provided space for these pages independent
of other missions of the FreeNet, but in no way denies or concurs
with anything on them. Our opinions are our own and your's are your's.
Last updated 6/26/95 5:40pm
_________________________________________________________________
From the research team:
Guest Page: Marketing Pornography on the
Information Superhighway (p2 of 2)
We commend the Georgetown Law
Journal for having the courage and foresight to publish our study,
Marketing Pornography on the Information Superhighway. The editors
are leading a new generation of legal scholars into the vast, unexplored
realm of Cyberspace. They are among the first to recognize the enormous
challenges that the legal community faces, as a result of rapidly
changing, and increasingly complex, technologies.
This article will
be published in Volume 83, Issue 5 of the Georgetown Law Journal.
You can direct order this entire article from the Georgetown Law
Journal at (202)662-9240.
_________________________________________________________________
Current plans for pages include the Introductory text from this
article and the conspiracies which have reached the ears of the
researchers.
_________________________________________________________________
This page is under significant construction. Watch this space over the course of the next couple weeks to see its development.
Guest Page: Marketing Pornography on the Information Superhighway
(p3 of 2)
media.1029.279: jim thomas (jthomas) Wed 28 Jun 95 11:11
ped writes:
> Today's Washington Post story followed Mike's line that the Rimm study is
> *only* about adult BBSs, and ended up building their criticism of TIME
> around a glaring error.
ped, clarify. The above reads as if you're claiming that it is MNEMONIC who made the glaring
error. If that is your point, please re-read #26, above, in which
the authors themselves say, explicitly, that the study, of which
you presumably have a copy, is about BBS material:
"This article
analyzes only the 450,620 images and descriptions for which complete
download information was available."
Those images were the BBS images,
ped. Or was there another article that to which you have access?
Or, does the abstract not match the article? Or, did you simply
not bother to read carefully when doing a story with such profound
implications? ped, you owe us a major clarification. I suspect you
also own mnemonic an apology.
media.1029.280: jim thomas (jthomas)Wed 28 Jun 95 11:18
In another forum, Aaron and Philip had the following
exchange Tuesday:
>In article <3so55l$ok@park.interport.net>, Aaron L Dickey > wrote:
>>
>> I remain hopeful Phillip will shed some light on just why this undergrad
>> term paper was given such incredible legitimacy by TIME's editors.
>This was hardly an undergradate term paper. Let's revisit this subject
>when you've seen the study. There's a web site under construction. I'll
>post the URL when I get it.
Philip, Aaron was perhaps a bit
glib in calling it a "term paper, but I suspect that you fully well
understand that the issue is *methodological* competence. Your patronizing
comment to "revisit" after seeing the study" is astonishingly disingenuous,
especially after all that's been written here prior to your post
above:
1) The methodology isn't available because 2) the study isn't
available because 3) the study was embargoed because 4) Time had
an exclusive which means 5) YOU had an exclusive that 6) earned
you some space on Night Line to 7) talk about "smut".
You're part
of the reason why we can't see the study, Philip. Let us see the
study, Philip, so we can "revisit" the issue. Or is the study currently
reserved only for a few chosen journalists and Ralph Reed?
This
incident is beginning to smell, ped, and you're right in the center
of it. Why will you not provide competent, published, recognized
methodolgists on the Well with a copy of it for independent assessment
of the method? I'm beginning to wonder what you're trying to hide.
media.1029.391: jim thomas (jthomas) Thu 29 Jun 95 22:54
> I still say, this is utterly irrelevant to the public issue. That
> is *my* agenda, mike. What's yours?
It's directly relevant to the public
issue to the extent that substantial factual and logical errors,
flip wording, extreme hyperbole, and the fact that ped conflates
Internet with "pornographic" BBSes reduces public dialogue on a
serious and complex issue to an abysmal smut teaser. It's relevant
to the public issue when he refuses to acknowledge his errors, thus
adding his credibility to claims that have, even in this short thread,
been convincingly refuted.
Some of us prefer the agenda of accuracy
to falsehoods in public debates. Sadly, Dave Hughes apparently does
not share that preference.
media.1029.395: jim thomas (jthomas)Thu 29 Jun 95 23:36
ped writes:
> I was writing about the Post article from memory. I had it wrong about where
> the error was. It did not say the study was only about adult BBSs. It did
> say that the study did not support TIME's thesis. That, I believe, is wrong.
I'm sitting here, stitched, swathed, sliced, diced, and pummeled
as the result of somebody totalling my car yesterday afternoon,
unluckily with me in it, but hoping to be psychically lifted with
at least some response from ped that responds intelligently to some
of the criticisms of his article. Instead, he offers a sentence
that maybe he or somebody better versed in grammar might parse so
it makes sense. Specifically, those "its" and "That" need a referent
or three.
ped, *you* did not say that the study was only about BBSes.
That claim is not "Mike's line."
That omission is part of the problem,
as cites from your piece have shown. Although the study alludes
to nearly one million items, Rimm acknowledges (p. 1854) that the
ARTICLE itself is based on only 292,114 IMAGE DESCRIPTIONS of ADULT
BBSes:
Of these (450,620), animations, text files, and images which
were either ambiguously described or not described at all, were
excluded. A total of 292,114 image descriptions [NOTE, ped, not
sex files, not graphics, not stories, but IMAGE DESCRIPTIONS] remained
and are discussed here (study, p. 1854)."
Did you notice, ped, that
while Rimm notes that the 917,410 items as "descriptions, short
stories, and animations," he shifts back and forth in describing
these variously as images, as GIFS (in one post), and as "descriptions?"
He seems deliciously inconsistent, using the higher, more encompassing
figure when it enhances the data set. It also wasn't clear how he
controlled for duplicate images with different discriptions. His
292,114 also appear to include "jokes" or prank images.
I'll go
be re-reading the study again, because many questions arose to which
I missed the answer the first time around. As somebody above wondered,
did the count occur after downloading or before? If nearly Rimm
downloaded nearly one million of 8.5 million items *before* the
count, and if his were not excluded, then he alone accounts for
about eight percent of the downloads. Perhaps he clarified this,
and if so, I'll cite the clarification when I find it.
media.1029.397: jim thomas (jthomas) Fri 30 Jun 95 00:03
In a private post, I chided Donna for raising the ethics issue. Judging from what I'd read here,
I was not convinced that a "human subjects" study necessarily required
a full review. I withdraw that reservation. There are several places
where explicit questions of ethics arise, and had I been on a human
subjections review committee, I would have objected if the "methodology"
were written into the proprosal as implemented in the study. Here
are just three:
- (p 1880, in describing how file lists were obtained):
In these instances, members of the research team either screen captured
the "all files" list in double line format, persuaded the sysop
to provide the list privately.
Question: How were the sysops "persuaded?" Were they aware of
the purpose of the request? Perhaps the requests were legitimate,
but the question should still be addressed in the text. This is
something that any competent peer review should have raised.
- p. 1995, in describing how access to a BBS's user demographics
was obtained:
The research team obtained demographics from several
leading "adult" BBS which indicate the age, sex, and city of origin
of subscribers. These demographics were based on verified credit
card information and were obtained either directly from the logfiles
of "adult" BBS or various methodologies developed by the research
team programmers.
In a footnote (#88, p 1995), Rimm writes:
Because of the sensitive nature of the consumer data, the research
team consulted privacy experts before submitting this article
for review and publication. It was decided that once the demographics
were tabulated and independently confirmed by two reviewers,
the names of all consumers would be permanently deleted from
the Carnegie Mellon database. This was done to protet the privcy
of consumers in the United States and abroad.
I'll leave it to others to identify
the self-evident questions.
- Appendix D (pp 1926-1934): Rimm identifies
the countries, States, and cities from which users accessed the
BBSes. I'd be astonished if the sysops would release such information
to the public, but whether they did is irrelvant. Given the sensitive
nature of such information, such a list should not have been compiled.
One of the first principles of ethics in social science research
is protection of subjects, and publishing such information, which
could easily be summarized in a table, crosses the line (in my view)
into explicitly unethical practice.
media.1029.398: jim thomas (jthomas) Fri 30 Jun 95 00:19
An aside that somebody already raised above:
There is a curious repition of a few words that one doesn't normally
see in a study. Rimm continually refers to himself as the "principal
investigator." Now, ordinarilly, this might not be significant.
In the "study," however, the term appears sufficiently often that,
if I were reviewing the paper for a journal, I would raise the issue
about self-promotion. This would seem a trivial quibble if Rimm
did not continuously allude to the project as "The Carnegie Mellon
study." Evoking association with Carnegie Mellon symbolizes an aura
of competence and respectabilty that, in this case, isn't warranted.
It's also curious that, with such an apparently substantial set
of "colleagues" and assistants, that Rimm is sole author. These
may seem in group projects to include others who had add up to diminish
an author's (and a study's) credibility.
media.1029.433: jim thomas (jthomas) Fri 30 Jun 95 15:32
((sorry for the length of these posts,
but they can't readily be done in a single screen))
Beating me to
a point I was going to raise later tonight, Mike slips in:
> This
> has created a thriving underground market for 'private collections' and
> anonymous ftp sites on the Internent, which cannot be studied
> systematically.
As a practical matter, an ftp site can be studied
in the same "practical manner" as adult BBSes. Rimm's presumption
seems to be that identifiying them would be difficult. If the market
in fact is "thriving," identifying and gaining access should be
trivial. Perhaps Rimm was confused by the term "*anonymous* ftp
sites."
Rimm's footnotes (such as this one) illustrate the importance
of reading footnotes. Rimm makes an explicit claim as if it were
a demonstrable fact. He is saying that highly visible porn busts
have created an alternative market. This can be readily tested.
Let's call it the "displacement theory of 'porn' distribution."
The basic logic: As porn is actively suppressed in one area of cyberspace
(BBSes), it emerges in other (likely less public) areas.
Independent variable: porn suppression on BBSes
Dependent variable: emergence of "underground" markets
If the hypothesis holds, we could expect
to find covariance between our variables---in this case, as the
magnitude of the IV increases, so, too, does the magnitude of the
DV.
Now, we can argue about extraneous variables (eg, the dynamics
of any underground economy) as alternative explanations, none of
which Rimm f considers. But, there is another minor flaw here (other
than the fact that no data are presented): Rimm violates one of
the four fundamental premises of causal logic: A "cause" generally
comes before its "effect." There has been a thriving "underground
market" (effect) well before recent busts (cause).
Logic aside,
here's a clear case of a seemingly minor claim made without providing
evidence or pointers. Such small claims combine to reinforce the
central thesis of self-evident Net porn.
Let's take another example
of such logic, which we can call the "deprivation thesis:"
An unusual
amount of data was freely available from commercial "adult" BBS
primarily as a consequence of the evolution of the online industry.
Large commercial BBS such as America Online, Compuserve, and Prodigy
do not carry hard-core pornographic imagery, either for legal or
policy reasons. As a consequence, several thousand comparatively
small "adult" BBS have sprung up across the country (p 1861).
Today's
in-class excercise: Drawing from the first example, what's wrong
with this picture?
>Thus, it may be difficult for researchers to repeat this
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> study, as much valuable data is no longer publicly available.
See infra
^^^^^
> notes 89-95 and accompanying text."
This is the
best proactive dodge to hedge replication failure that I have ever
seen. "What I tell you is true, but if you can't replicate it, hey....that
ain't my problem."
Replication is a sine qua non of normal science.
A credible scholar would identify ways to replicate rather than
try to explain away a failure to replicate. In my eyes, this single
tidbit alone disqualifies the "study" as legitimate. It also sounds
warning bells about the quality of the data.
I'm also not sure which
data Rimm feels are "no longer available."
I also wish that somebody
had taught him that "BBS" is singular.
media.1029.487: jim thomas (jthomas) Sat 1 Jul 95 12:53
So many flaws, so little time.
jdevoto's
comment on the time required to examine the data items has been
addressed above by brock and me (o138, o158). She correctly notes
the extreme difficulty in both the logistics and analysis of over
1 million images (including Usenet files). Perhaps Rimm can adequately
explain the time factor. The fact is: He didn't, indicating the
sloppiness of the study.
If it's true (as the study itself suggests)
that a software coding program sorted through the texts as the primary
means of analysis, then this technique should be explicitly elaborated
in great detail. Despite a few vague allusions to its use, there
is virtually no explication of what is apparently a significant
component of the "analysis." This indicates sloppiness of the study.
It is a flaw that would sink it in any competent peer review.
Lest
the reading-impaired not know what this study is about, it's about
PORNOGRAPHY. Not erotica, not the on-line culture of sex, not about
an analysis of selected adult BBSes and newsgroups, but PORNOGRAPHY.
The word is used at least once on nearly every page, along with
"pornographer" and other explicit and often seemingly "scientific"
terms to alert the reader with sometimes subtle, sometimes explicit,
labels that connote and denote and value-laden and biased set of
images of who and what the topic is about.
media.1029.489: jim thomas (jthomas) Sat 1 Jul 95 12:54
Rimm makes so many unsubstantiated
claims that don't stand up to even modest scrutiny that I'm astonished
that anybody could take it seriously. Here's one small example that
would appear to enhance the study's significance, but in fact reveal
its flaws:
Two perspectives help inform this study of pornography:
that of the pornographer and that of the consumer. This study explores
pornography from the perspective of the pornographer by performing
a content analysis of the written descriptions provided by the pornographers.
It explores pornography from the perspective of the consumer by
examining consumer habits for various classifications of images
(p. 1855).
Rimm seems quite unfamiliar with the voluminous literature
on the methodology of content analysis, which likely explains why
so much of the study is flawed by weak data and incomprehensible
over-generalizations (Later, I'll list a few as time allows).
More
importantly, the claim that the study somehow captures the point
of view of "the pornographer" and the "consumer" is recklessly wrong.
Even an undergrad with minimal methods training would recognize
that seeing something from an informant's point of view is an ethnography.
This is no ethnography. Worse, there is NO DATA from a consumer's
point of view. Not one interview, not one survey, not one narrative.
Same for "pornographers." One cannot tell from downloads anything
about a "point of view". One cannot tell Robert Thomas's point of
view by imputing meanings to court records (from which most of the
AA BBS data came from according to the footnotes). This study is
NOT from the "other's" point of view--it is from Rimm's point of
view.
The operationalization of "consumer" reflects sloppy and inappropriate
conceptualization. Rimm defines a consumer as anybody who downloads
a file or accesses a "pornographic" newsgroup, but this tells us
nothing about intent, motive, or use of the downloader. What do
people do with the downloads? Do "consumers" actually like such
material, or are they merely curious? How many "consumers" access
Usenet, for example, without downloading? Anybody who has been on
an alt.binaries group knows that much, perhaps even most of the
message traffic is not pornographic, but is instead discussions,
flames, or general chat. To understand the "consumers" point of
view, at least some of this data could have been presented. After
all, Rimm claims to have accessed Usenet posts and files through
screen captures. This "method" and Rimm's claims about just won't
hold up.
This is not a minor point. It is a fundamental methodological
claim intended to elevate the study's significance. That such a
fundamental claim is untrue is further evidence the Rimm either
doesn't know or doesn't care about "science."
media.1029.492: jim thomas (jthomas) Sat 1 Jul 95 13:00
There is one minor, seemingly
inconsequential, sentence that I missed on my first time through.
Rimm indicates that "adult" BBS operators do not mind having their
wares publicly redistributed:
While such data was not available
for this study, most BBS sysops with whom members of the research
team "chatted" indicated no objection to their customers postng
their images on the Usenet (1875-76).
Here are a few things I found
troubling about this single sentence:
- Terms like "many" or "most"
are used throughout to bolster claims. Here, however, the key phrase
is "most...with whom members..'chatted'"... This "most" could be
a small percentage of the total sample. Perhaps it's not, but it's
the type of data-free generalization that casts doubt on the data's
credibility.
- What strikes me most, however, is "chatted." To
me, it suggests that contact with BBS sysops may have been haphazard,
unsystematic, and even sparse. Perhaps it's just the sloppy wording
and a clarification would dispel my concerns. But, the line suggests
that contact with sysops was limited to sketchy on-line chats. If
not, it should be clarified. If so, then what are the implications
for how the "team" obtained the data? Was there an interview schedule,
a pre-determined set of statements, forms, or other documents? What
does it mean to say "while such data was ((Rimm does not realize
that the term "data" is plural, a mistake he repeats)) not available
for this study? If the data were not available, then why does he
make the claim?
If the methodology were self-evidently competent
(which it's not) or reasonably elucidated (which it's not), such
jarring questions would not arise. These are no longer small points,
because they represent growing evidence of either serious intellectual
deception or gross scholarly incompetence.
media.1029.500: jim thomas (jthomas) Sat 1 Jul 95 14:43
Steven writes:
[quoting from Rimm]:
> "These and other findings may assist policymakers and others
> concerned with the future of Cyberspace to make informed decisions,
> with reliable data, about the evolving Information Superhighway."
> When I asked Rimm how he felt that his study would invetably be sucked
> into the policy debate, he said that was not his intention and he
> had no desire for it to be used that way......
If accurate,
Steven's info reflects Rimm's astonishing disingenuousness, perhaps
outright dishonesty. Rimm seems not to know what he wrote in the
study that emphasizes policy implications, including:
The study's
findings may have serious implications for legal theory and public
policy related to pornography. Among the ultimate findings of this
study are that digitized pornographic images are widely circulated
in all areas of the country and that, due to market forces, digitized
pornographic images treat themes such as bestiality and pedophilia,
which are not otherwise widely available (p. 1857).
[after discussing
that "childporn" generally requires the depiction of an actual child
(fn #20), Rimm writes:]
If technology advances, as it surely will,
to allow the creation of pornographic images that do not depict
actual children, this justification for prohibiting the dissemination
of these images may no longer be compelling (p. 1858).
"The widespread
availability of pornography on computer networks may have a profound
effect on those who wish to utilize the emerging National Information
Infrastructure for non-pornographic purposes (p. 1858).
In the final
analysis, if the application of digital imaging to pornography has
any recurring theme, it is that traditional constitutional and law
enforcement assumptions and conclusions must adapt to a new technology
(pp 1859-60).
In addition to the study of pornography distribution,
comsumption, and regulation, this study has ramifications for other
fields of research and regulation (p. 1861).
An analysis of pornography
in cyberspace provides a fascinating case study of many of the computer
network-related legal and technological issues confronting businesses
and policy makers today (p. 1862).
Numerous other passages denote
the intended policy ramifications of this study. While Rimm may
not have chosen to reveal to Steven his stance on current restrictive
legislation, I find the conclusion of the "study" revealing. Under
the subhead "F. The MARQUIS de Cyberspace," Rimm notes that law
enforcement efforts have done little to discourage consumers or
Thomas from their enterprise. Careful readers will note that in
a previous section, Rimm claimed that this specific highly visible
case "caused" the rise of erotica on anonymous ftp sites, suggesting
a displacement from conventional to less visible "underground" access
sites. While it is possible to reconcile this conclusion with the
earlier claim, it's not possible to decipher from the existing text,
and is one of the oh-so-many examples of confused thinking and discourse.
But, I digress from my point, which is that the final paragraph
may reveal Rimm's position for those who skipped pages 1849-1911:
Men of considerable intelligence have paid homage to sade, admiring
his urivaled, demented imagination. Yet for all their efforts, Sade
and his disciples pushed pornography only as far as the printed
word allowed. Two centuries of technological innovations--the photograph,
the digital image, the scanner, computer bulletin boards, computer
networkds--passed before Robert Thomas would present us with Amateur
Action BBS, a hi-tech rendition of _The 120 Days of Sodom_. The
Marquis, it seems, has finally been topped.
This ain't no detached
door-closer, folks.
I can't but wonder whether the inclusion of
cities of "porn consumers" in Appendix D, which to my mind is a
reprehensible breach of research ethics that should be brought to
the attention of CMU, isn't a subtle policy-relevant scare to alert
small communities of PORN AT THE DOOR, thus expanding the base of
anti-erotica opposition amongst the grass roots.
Given the study's
explicit policy promotion and the manner in which it was shopped
as a media event, there is no way that Rimm can convincingly claim
that he had no desire to see the study used that way. This is a
policy-oriented document, and the slant isn't subtle.
media.1029.530: jim thomas (jthomas) Sat 1 Jul 95 23:15
(I apologize for the length--well
in excess of 100 lines. If people would prefer such stuff hidden,
holler).
I've been stumped by the "Carnegie Mellon" study's Usenet
discussion (pp 1865-76), and subsequent readings don't help. If
the following cannot be adequately explained, it is a fatal flaw
that destroys any credibility of the study's methodology and casts
doubt in the intellectual integrity of Rimm, CMU (this is, after
all, *THE* Carnegie Mellon Study), Rimm's advisors, and other participants.
On p 1865, Rimm states:
All of the Usenet newsgroups with the prefix "alt.binaries" were
examined each day for a period of seven days, *BEGINNING
^^^^^^^^^^^^^
SEPTEMBER 21, 1994 AND ENDING SEPTEMBER 27, 1994* [emphasis added].
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Only the pictures hierarchy was included, because sound files and
other non-picture binaries are not the subject of this study. The
number of new images posted each day was tabulated for both pornographic
and non-pornographic newsgroups. Because thse numbers were tabulated
based on screen captures that indicate the date, name, size, and
origin of the post, a random sampling of twenty images per day (ten
suspected pornographic and ten suspected non-pornograpic) was downloaded
to confirm the legitimacy and content of the images described (p
1865).
Of the next two paragraphs, the first notes that images are
occasionally posted to non-binary newsgroups, and the second introduces
a computer system where additional Usenet data were acquired (p.
1865-66. The bulk of p 1866 consists of two footnotes).
In the next,
third, substantive paragraph of text:
*AFTER* examining the prevalence and popularity of pornographic
imagery on the Usenet, the research team was interested in determining
the origins of such imagery. The largest selection of sexual imagery
was discovered on the following five Usenet newsgroups(32):
alt. binries.pictures.erotica
alt.binaries.pictures.bestiality
alt.sex.fetish.watersports
alt.binaries.pictures.female
alt.binaries.pictures.tasteless
*BETWEEN APRIL AND JULY OF 1995*, the research team downloaded
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
all available images (3254) from thse five newsgroups. The team
encountered technical difficulties with 13% of these images, which
were incorrectly encoded or incorrectly uploaded by the poster.
This left a toal of 2830 images for analysis. The images were then
decoded from the format used for transmission into image files (pp
1866-1867].
(Footnote 32, cited above, reads: "32. These [Usenet
groups--jt] were the largest available at the research site. (p.
1867).
There are numerous minor questions that could be raised about
the silences in these passages, including: How were controls for
duplicate, resent, or prank image files implemented, if at all?
Was it posts or complete files that were counted? The latter is
crucial, because a single image may require ten or more files to
complete. (Coincidentally, I was on alt.binaries.pictures.erotica
today, and of the roughly 1,650 posts, one of the files was a 41-post
executable that constituted about 2.5 pct of the message volume).
How was the random sampling of 20 files a day done? What were the
results? Why should we take this data seriously?
Those aren't my
questions.
This one is:
Where did that list of five sites come from?
Even a superficial reading of the text seems clear: The methodology
discussion begins by explaining how binaries groups' posts were
tabulated *OVER SEVEN DAYS IN SEPTEMBER* (p. 1865). Rimm continues
(p. 1866) by stating that *AFTER* examining the prevalence of "pornographic
imagery on the Usenet" they were interested in determining the origins
of such imagery. They identified five.
QUICK SUM OF THE PLOT SO FAR FOR THOSE WHO ARE SKIMMING:
The section began with a discussion
of methodology for tabulating the number of new porn images a day
posted to Usenet. Then, it seems clear, the authors used that data
to identify the five newsgroups with the most images, from which
they then downloaded 3,254 images.
I've re-read the text many times,
and there is no way I can find any other interpretation: The 7 day
tabulation makes it appear that it was used as the basis for ranking
the five top Usenet "porn" groups. Granted, the vagueness in wording,
logic, and details leaves gaps, but the implication seems, to me,
crystal clear:
They tabulated the newsgroups in SEPTEMBER '94, AFTER
WHICH they downloaded from the top five between APRIL-JULY '94.
Is there a typo? Is there an astonishing inability to clearly articulate
what was actually done? Did somebody edit out a segue between sub-sections?
Was a crucial methodological section inadvertently omitted by the
publisher? Is this *really* a study?
Ah, but now we get back to
that footnote #32: "These were the largest available at the research
site. (p. 1867). Ok, so now we have another possible explanation:
This study isn't about the domain of Usenet, but rather about the
newsgroups available at a single site. This, of course, is fine---but
that's not what the text leads the reader to believe the section's
topic is about. And, if the data are simply the five largest groups
on the research site rather than in the Usenet domain, then why
bother with the introductory methods material on p. 1865 that make
it appear that a systematic examination OF ALL USENET NEWSGROUPS
(p 1865) occured to generate that list of five? And, of course,
the enevitable question: Why isn't there a methodological discussion
describing controls to assure that the images/posts were non-duplicates,
non-jests, well, you know the drill.
I'm certain that Rimm (or others)
may be able to explain this seemingly devastating incongruity. But,
it doesn't really matter: A competent study should on no account
raise so many basic questions that can only be answered outside
of the text itself.
media.1029.532: jim thomas (jthomas) Sat 1 Jul 95 23:45
P. 1867 of *THE* Carnegie Mellon study indicates that there
were 3,254 available images amongst the five top newsgroups over
a 122 day period (April/July).
This means:
650.8 = avg number of images (assuming the article actually MEANS images and not posts
or files)
5.33 = avg number of new files per day (650.8 / 122 days;
This assumes four full months. Even if period was only from Apr
30 thru July 1, it would still be only about 10 new files per day
per group on average)
Now, buried in a footnote (#31 on p. 1866)
is this: "For instance, forty new images were posted to alt.binaries.pictures.erotica
(a.b.p.e.) each day...."
The source of the factoid isn't given,
and the time-frame of the statistic isn't given, but that's minor.
a.b.p.e. is one of the five listed Usenet groups under analysis.
If that 40-a-day-new-image tidbit is correct, how does it compare
with the averages given above? If both the factoid and the averages
above are correct, that means a.b.p.e. would consist of between
2520 to 4,880 images/posts over between April/July, depending on
whether the period was about 122 days (1 apr/31 july) or 63 days
(30 apr/1 july).
Such a disproportionate cluster skews the imagery
of the Carnegie Mellon study's descriptive statistics: Take out
one group, and the magnitude of "porn" diminishes dramatically.
There are just too many questions and discrepancies to allow anything
in the Carnegie Mellon study taken seriously.
media.1029.535: jim thomas (jthomas) Sun 2 Jul 95 00:30
In Table 1, "Top Forty Usenet Newsgroups at a University Studied"
(pp 1870-71), the user base
is presumably 4,227 (p. 1865, which should be moved closer to the
table if it refers to the same system). The serious failure to explain
the column headings aside:
- The system receives about 3,600 of
the 14,000 (p.1862) Usenet groups. That constitutes about 25.7 percent
of all groups. Is this the site alluded to earlier in the article
(fn #32, p 1867)? Such a small percentage of the total groups is
not sufficient for extrapolating the meanings that the Carnegie
Mellon study imputes to the figures. If the remaining 10,000+ newsgroups
were added to the data, it is likely that the 2.88 pct of the newsgroups
available to users of this system would evaporate.
- Astonishingly,
there is neither a comp nor news group among the top 40. The lowest
(fortieth) group on the list has 99 readers. I can't believe that
at a private university (especially if it is Carnegie Mellon), at
least 99 of 4,227 users wouldn't read a comp group. Sorry, Carnegie
Mellon, either your data are wrong or the implications your study
derives from it are deceptive.
- The top forty newsgroups in Table
1 diverge so wildly from the traffic summarized by periodic arbitron
stats that the Carnegie Mellon team and principal investigator are
required simply as an intellectual matter to clarify the discrepancy.
That the study emphasizes this table as significant evidence in
establishing the Net-porn thesis is reprehensibly deceptive given
the atypicality of reading and access patterns.
- When I raised
this earlier today with Donna, she happened to have an undergrad
paper by one of Tom Novak's students, who did a study of newsgroups.
The student found that the most popular group at Vanderbilt was
alt.irc. As I recall, Donna indicated that the highest sex group
on the list was ranked 29th. Perhaps she can elaborate further.
Slice it as you will, the data in Table 1 of the Carnegie Mellon
study do not reflect typical Usenet patterns. This destroys the
table's credibilty. That the principal investigator makes no mention
of this atypicality and makes no mention of obvious discrepancies
is something I'll charitably attribute to undergraduate naivity
rather than intentional deception.
There is, of course, another
possibility. If this user base is in fact from Carnegie Mellon,
it's possible that folks there are so obsessed with erotica that
they'll even find a way to view it under the guise of research.
media.1029.537: jim thomas (jthomas) Sun 2 Jul 95 01:07
Another
observation about Table 1 (pp 1870-71):
Its limitations are described
five pages earlier (fn 30, p 1865) in a way that suggests that it
underestimates rather than overestimates "pornography." No mention
is made of any of the issues raised in the previous posts:
30. Each
of the data sets [Usenet data available via ftp and the university
system--jt] has limitations. The university data, while perhaps
unique in that it offers an accurate, if not exhausitive, picture
of the monthly access habits of an entire community, has two limitations.
First, 11% of the computer users in this study block the site statisticians
from monitoring their online activities. Second, some users have
multiple accounts and avoid detection by using a second account
to access the Usenet.
While there is no evidence to suggest that
Usenet and Internet users who block the monitoring of their accounts
access pornography more frequently than those who do not, one also
cannot assume that a notable difference does not exist. This is
especially true in the context of pedophilia and child pornography
consumption. Preferential molesters (i.e., pedophiles with a true
sexual attraction to children) frequently employ inventive mechanisms
to evade discovery, as discovery will likely lead to incarceration
[study cites a source for this claim--jt] (p. 1865).
When Donna
and I discussed this earlier today, she convincingly argued that
the second paragraph appears to smear (my term, not hers) those
who hide their accounts by associating them pedophilia. That second
paragraph strikes me as reprehensible innuendo.
Another maddening example of truth-by-assertion:
In broad terms, the research indicated
that pornographic newsgroups are accessed more frequently during
the school year than during summer recess. This suggests that, in
comparison to teachers, faculty, and staff, a disproportionately
large number of students access Usenet pornography (pp 1869-70).
Without some data, one can't be certain that Rimm knows the difference
between frequencies and ratios.
media.1029.579: jim thomas (jthomas) Sun 2 Jul 95 15:40
ped, let's start with your first response to
the Hoffman/Novak critique:
> 1) Hoffman and Novak describe the Rimm study as "an
> unsophisticated analysis of descriptions of pornographic
^^^^^^^^^^^^^^^^^^^^^^^^^^^
> images on selected adult BBSs in the United States" whose
> findings "cannot be generalized beyond this narrow domain."
^^^^^^^^^^^^^^^^^^^^^^^^
Then:
> Yet as they later acknowledge, the study did include an
> analysis of the number and source of a sampling of images
> available on the Usenet. The key phrase, ped, is "UNSOPHISTICATED ANALYSIS" whose findings
"CAN'T BE GENERALIZED." That Rimm later includes an "analysis of
the number and source...." on the net does not mean that Rimm's
analysis was methodologically, interpretively, or logically sound.
Read the title and even just the first few pages of the GLJ article.
THE STUDY IS OF BBSes, ped. Despite what you wrote in Time and in
your comment above, it's about BBSes. The Net "analysis" is "nasty,
brutish, and short," and the questions raised in previous posts
suggest serious flaws.
> That analysis showed that pictures
> from the adult BBSs were finding their way onto the public
> network.
It doesn't take a "study" to make a simple claim like, "commercial
BBS graphics are finding their way onto the Net. "All it takes is
one image. Further, the presence of accurate information in the
GLJ article does not override the existence of the other kind that
informed both your article the GLJ article.
> Since this is the issue that has some people so worried,
> why do Hoffman and Novak ignore it?
H/N are not doing a study of Net-porn. They are doing
a critique of the Time article and the study on which it was irresponsibly
based.
Nobody is disputing that Net-erotica exists. Nobody is denying
that Net-erotic doesn't pose problems or consitute a serious issue.
What some of us do dispute is whether either public discourse or
public policy are well-served by a fataly flawed "study" and a sensationalistic
Time cover story that lends what is arguably little more than a
polemic even minimal credibility.
media.1029.612: jim thomas (jthomas) Sun 2 Jul 95 21:55
Here are the top ten newsgroups in order of popularity
from the latest "arbitron" ratings:
------------
TOP 40 NEWSGROUPS IN ORDER BY POPULARITY
A companion posting explains the statistics and the algorithms that produced them.
+-- Estimated total number of people who read the group, worldwide.
| +-- Actual number of readers in sampled population
| | +-- Propagation: how many sites receive this group at all
| | | +-- Recent traffic (messages per month)
| | | | +-- Recent traffic (megabytes per month)
| | | | | +-- Crossposting percentage
| | | | | | +-- Cost ratio: $US/month/rdr
| | | | | | | +-- Share: % of newsrders
| | | | | | | | who read this group.
V V V V V V V V
1 1200000 3270 90% 41 0.6 21% 0.00 5.9% news.announce.newuser
2 340000 1062 83% 4357 6.6 12% 0.01 1.9% comp.lang.c
3 330000 1643 51% 7737 47.1 30% 0.06 3.0% alt.sex.stories
4 320000 1041 80% 33891 50.4 26% 0.11 1.9% misc.jobs.offered
5 250000 1114 56% 4092 5.4 3% 0.01 2.0% alt.tv.simpsons
6 240000 703 88% 155 1.6 12% 0.01 1.3% news.announce.newgroups
7 230000 715 81% 5475 7.3 23% 0.02 1.3% comp.lang.c++
8 210000 751 72% 32 0.1 3% 0.00 1.4% rec.arts.startrek.info
9 200000 713 71% 5097 7.4 11% 0.02 1.3% rec.arts.startrek.current
10 190000 664 73% 606 2.0 1% 0.01 1.2% rec.food.recipes
--------------------------------
Using the methodology
of the Carnegie Mellon study, it's indisputable that the nation
is obsessed with C language. I've surveyed over 1,400 data items;
it's obvious that consumers of Usenet information would rather program
than beat off. This from the point of view of the consumers themselves.
The current top 40 list is so divergent from Table 1 of the Carnegie
Mellon study that failure to explain the discrepancy for such a
prominent table is an error that, itself, would be sufficient to
reject the study for publication and subvert the credibility of
its other findings.
media.1029.621: jim thomas (jthomas) Sun 2 Jul 95 22:25
Note: It's tricky to pull together Usenet stats for any
kind of analysis, and although Rimm mentions this, he doesn't address
it methodologically or as a caveat in his own discussion. Nor does
he let this stop him from making unfounded generalizations (Tables
1 and 2).
A few examples of problems: I don't recall ever seeing
even an approximation of the full range of Newsgroups listed. Rimm
claims that it's around 14,000. The "arbitron" ratings usually list
about a tenth of that. The actual number of active groups is likely
somewhere in between.
Some groups aren't always listed. A specific
example: Cu Digest is one of the rare groups that, for historical
reasons, is read as both a comp and an alt group. Sometimes only
comp.society.cu-digest is listed, and sometimes (as this month)
only alt.society.cu-digest. Sometimes, both groups are listed. The
comp readership is substantially higher than the alt readership,
and if only one is listed, it would dramatically skew any assessment
of readership.
I'd suspect that excluding the entire population
of Newsgroups would inflate the percentage of erotica-oriented groups
(on the assumption that the less-used groups are specialized, non-erotica).
Rimm makes no effort to control for partial data.
On p 1873 of the
study, Rimm notes that in his "analysis," 20.4 percent of the monthly
Usenet posts in the top forty newsgroups (from his Table 2) are
"pornographic." He simply does not have the data to support this.
Period! Another problem with analyzing Usenet activity is that you
can't assume message content by the group's name.
The problems with
this study are insurmountable. The problem is not that there are
errors, and it's not that the errors are often serious. It's that
the fundamental assumptions and procedures that guide the study
are so flawed that there's they subvert the credibility of even
the legitimate points.
With proper guidance and considerably more
care, Rimm might have used his resources to produce an interesting
bit of work. Instead, he has produced an embarrassment to which
he attaches CMU's name and ped's credibility.
media.1029.624: jim thomas (jthomas) Sun 2 Jul 95 23:05
> Is it possible that Rimm's Table 1 is based on traffic volume
> rather than article count? I've found that's a common error in
> these sorts of studies.
jef,
Rimm's Table 1 (top forty at presumably CMU) is taken from (CMU)
local sources. The column headings aren't really explained, and
it's not clear from what I can decipher precisely how they were
obtained.
Table 2 is the standard "top forty/worldwide" popularity
figures based on estimated Usenet readership extrapolated from the
arbitron sample.
Table 3 is the "top forty newsgroups in order of
traffic volume, worldwide" arbitron. He uses Table 3 to introduce
another policy implication (contradicting the reported claim that
he didn't intend it for policy). He writes:
Policymakers might be
equally interested in comparing the cost of pornographic and non-pornographic
newsgroups. the primary cost factor in maintaining a newsgroup is
the amount of storage space required. Image files require considerably
more storage space per image posting than a story or comment. The
following chart lists the top forty Usenet newsgroups by volume:
(Then comes Table 3, top 40 in order of traffic volume)
The discussion
of Table 3 is short:
In the top forty newsgoups, the percentage
of total cost used for pornographic newsgroups is 34.2%. Of total
space taken for image newsgroups, 76% of the space is used for pornographic
images. Of total space taken for text newsgroups, 9.8% are pornographic
(p. 1874).
He doesn't explain how he obtained that 34.2% figure,
unless it's from the cost ratios provided by arbitron (I haven't
whipped out the calculator for Table 3).
There is considerable room
for honest debate over space and cost constraints in carrying high-volume
groups. There is no room to quibble over the fact that he seems
to have conflated text and binary posts in his calculation, counting
text as image. The degree to which this would alter his figures
isn't clear. What is clear is that he not only conflates disparate
types of material, but--worse--he provides no way for the reader
to sort it out or make an independent assessment.
media.1029.627: jim thomas (jthomas) Sun 2 Jul 95 23:19
A marvelous line from footnote
30 (pp 1865-66):
The research team has also mounted a continuous
and persistent effort to obtain similar data [similar to his CMU(?)
data in Table 1--jt] from other universities. In most instances,
the universities contacted indicated that they did not compile such
data, while others were uncooperative. Still, there is no
^^^^^^^^^^^^^^^^^^
reason to believe consumption at the university studied differs from
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
that of other universities from which pornographic Usenet newsgroups
can be accessed (p. 1866).
One reason to suspect differences might
be the dramatic difference between Table 1 and Table 2. Although
the populations are different, the difference provides a exceptionally
strong reason to at least question the typicality of the studied
site.
These (like many of the other criticisms) aren't arcane questions
of epistemology and methodology. They're simply observations of
weak prose and questionable figures that don't add up to a competent
study.
|