Key Concepts and Commands
\(F(y) = \beta_0 + \beta_1 x_1 + ...\)
\(y = \ln(count): 1, 2, 3, \text{etc.}\)
Think about IRR’s, predicted counts, non-linearity
So much of categorical data analysis depends upon arguments for “functional form”. When do we think these arguments are valid?
Historical Examples of Count Data 🐎 ☎️ 🏥
- 🐎 Deaths from horsekicks in the Prussian Army
- ☎️ Calls to a call center (business, crisis hotline, etc.)
- 🏥 Arrivals at the Emergency Room
From the Stata documentation:
“Poisson regression fits models of the number of occurrences (counts) of an event. The Poisson distribution has been applied to diverse events, such as the number of soldiers kicked to death by horses in the Prussian army (von Bortkiewicz 1898); the pattern of hits by buzz bombs launched against London during World War II (Clarke 1946); telephone connections to a wrong number (Thorndike 1926); and disease incidence, typically with respect to time, but occasionally with respect to space.”
Theorizing about the Poisson 🐟🐟🐟 🌲🌲🌲
The Poisson distribution is a modified form of the binomial distribution that is useful for the analysis of phenomena characterized by discrete, rare events. For example, in a study of the distribution of a rare plant among a number of quadrats, a majority of plots may contain no specimens, a smaller number a single plant, and still smaller numbers two, three, or more plants. If a single plant per quadrat is expected, the mean µ = 1 and the “0” and “1” classes occur at 37% each, the “2” class at 18%, the “3” class at 6%, and larger classes take up the remaining 2%. The characteristic test for a Poisson is that the variance equals the mean, which in the plant example means that the rare plant is randomly distributed. Note, that, In the distributions above, as the mean \(\mu\) increases towards 10, the distribution approaches normality.
The Poisson may be used to evaluate whether events occur independently in time as well as space. In a classic example, Bortkiewicz (1898) studied the distribution of 122 men kicked to death by horses among ten Prussian army corps over 20 years. In most years in most corps, no one dies from being kicked; in one corp in one year, four men were kicked to death. Does this mean something was amiss in this particular corp? Analysis indicates that the observed frequencies conform quite closely to the expected Poisson frequencies, and that the mean and variance are almost identical, as expected. The corp was just “unlucky” in the sense that it is in the extreme tail of an ordinary run of events.
http://www.mun.ca/biology/scarr/smcPoisson_distributions.html
Reprise of Normal Distribution
Normal distribution:
\[ P(x) = \frac{1}{\sigma \sqrt{2 \pi}} e^{-(x-\mu)^2/(2 \sigma^2)} \]
Standardized Normal Distribution:
\[ P(z) = \frac{1}{\sqrt{2 \pi}} e^{-z^2/2} \]
2 parameters:
\[ x \sim N(\mu,\sigma) \]
. clear all // clear all for simulated data
. set obs 1000 // number of observations
Number of observations (_N) was 0, now 1,000.
. generate x = rnormal() // normally distributed random variable
. histogram x, title("Normally Distributed Random Variable") scheme(michigan)
(bin=29, start=-3.0031285, width=.20304677)
. graph export myhistogram.png, width(500) replace
file /Users/agrogan/Desktop/GitHub/newstuff/categorical/count-regression/myhistogram.png saved as
PNG format
Poisson Distribution
\[ P(Y=y) = e^{-\lambda} \frac{\lambda^y}{y!} \]
A Poisson with large lambda looks very similar to a normal distribution.
. generate w = rpoisson(.5)
. histogram w, title("Poisson Distributed Random Variable") scheme(michigan)
(bin=29, start=0, width=.13793103)
. graph export myhistogram2.png, width(500) replace
file /Users/agrogan/Desktop/GitHub/newstuff/categorical/count-regression/myhistogram2.png saved
as PNG format
National Survey of Children’s Health (2018)
The data are an extract of the National Survey of Children’s Health, 2018. The data contain information on children’s exposure to various Adverse Childhood Experiences (ACEs) and their demographic characteristics.
. use "../predict-and-margins-substantive-example/NSCH_ACES.dta", clear
. describe
Contains data from ../predict-and-margins-substantive-example/NSCH_ACES.dta
Observations: 30,530
Variables: 23 20 Oct 2020 14:50
───────────────────────────────────────────────────────────────────────────────────────────────────
Variable Storage Display Value
name type format label Variable label
───────────────────────────────────────────────────────────────────────────────────────────────────
sc_sex byte %30.0g sc_sex_lab
Sex of Selected Child
ace3 byte %30.0g ace3_lab Child Experienced - Parent or Guardian Divorced
ace4 byte %30.0g ace4_lab Child Experienced - Parent or Guardian Died
ace5 byte %30.0g ace5_lab Child Experienced - Parent or Guardian Time in Jail
ace6 byte %30.0g ace6_lab Child Experienced - Adults Slap, Hit, Kick, Punch
Others
ace7 byte %30.0g ace7_lab Child Experienced - Victim of Violence
ace8 byte %30.0g ace8_lab Child Experienced - Lived with Mentally Ill
ace9 byte %30.0g ace9_lab Child Experienced - Lived with Person with
Alcohol/Drug Problem
ace10 byte %30.0g ace10_lab
Child Experienced - Treated Unfairly Because of Race
ace1 byte %30.0g ace1_lab Hard to Cover Basics Like Food or Housing
sc_race_r byte %48.0g sc_race_r_lab
Race of Selected Child, Detailed
sc_racer byte %31.0g sc_racer_lab
Race of Selected Child, Recode
higrade byte %61.0g higrade_lab
Highest Level of Education among Reported Adults
depress byte %9.0g RECODE of k2q32b (Depression Currently)
ace1R byte %9.0g RECODE of ace1 (Hard to Cover Basics Like Food or
Housing)
ace3R byte %9.0g RECODE of ace3 (Child Experienced - Parent or
Guardian Divorced)
ace4R byte %9.0g RECODE of ace4 (Child Experienced - Parent or
Guardian Died)
ace5R byte %9.0g RECODE of ace5 (Child Experienced - Parent or
Guardian Time in Jail)
ace6R byte %9.0g RECODE of ace6 (Child Experienced - Adults Slap, Hit,
Kick, Punch Others)
ace7R byte %9.0g RECODE of ace7 (Child Experienced - Victim of
Violence)
ace8R byte %9.0g RECODE of ace8 (Child Experienced - Lived with
Mentally Ill)
ace9R byte %9.0g RECODE of ace9 (Child Experienced - Lived with Person
with Alcohol/Drug Problem)
ace10R byte %9.0g RECODE of ace10 (Child Experienced - Treated Unfairly
Because of Race)
───────────────────────────────────────────────────────────────────────────────────────────────────
Sorted by:
Generate Count of Aces
. egen acecount = anycount(ace*R), values(1) // generate count of ACES
. histogram acecount, discrete scheme(michigan)
(start=0, width=1)
. graph export myhistogram3.png, width(1000) replace
file /Users/agrogan/Desktop/GitHub/newstuff/categorical/count-regression/myhistogram3.png saved
as PNG format
Over-Dispersion
Due to population heterogeneity (diversity, variation), variance may be \(>\) mean. This is often empirically the case.
\[ \text{var}(y) > \text{mean}(y) \]
\[ y \sim Poisson(\mu) \]
\[ \ln(\mu) = \beta_0 + \beta_1 x + \text{offset} + \text{dispersion} + etc. \]
Predicted Values
. predict yhat
(option n assumed; predicted number of events)
. histogram yhat, scheme(michigan)
(bin=44, start=.50284678, width=.05128577)
. graph export myyhats.png, width(1000) replace
file /Users/agrogan/Desktop/GitHub/newstuff/categorical/count-regression/myyhats.png saved as PNG
format
