Psychometrics

A new look at student performance…

A new look at student performance…

This article, reporting on the data of new hires at Google, should teach us a lesson.

Google is an expert in data collection, analysis, and the business of technology (which is pretty much the future of the modern world’s economic growth and is still barely mentioned in any school curriculum). They’ve reported that a new hire’s GPA and their success on Google’s legendary in-interview brain-teasers are NOT predictive of job performance.

Further, another study that I’ve just learned of shows that there is a score threshold for predicting success in scientific subjects, specifically math and physics, that does not exist in any other test for any other major. So maybe standardized tests are great… but only for technically minded students.

How long will it take us to integrate this new information into the way we educate our children? Considering the current stagnation of progressive, forward-thinking policy shifts, it will probably take decades.

Let’s just consider the possibility of breaking the strict guidelines of our current curriculum, allowing students to express their interests and creativity in an environment that fosters ideas and the analyzation of current events and breakthroughs. I would never suggest removing all levels of testing and quality control measures like grades… but this continuous discussion of “accountability” (which is typically aimed squarely at teachers) should start being refocused onto the policy maneuvers that brought us to the deficiencies we witness in our public education system.

Read More

Testing in Education: An Identity Crisis

Testing in Education: An Identity Crisis

Are you for success of the whole? Or success of the individual? Is there a way to measure both, or just one?

Is education the filling of a bucket? Or the ignition of a flame? Obviously the amount that is in the bucket would be an easy measurement, but what about the “motivation” to increase the size of your bucket? How do you measure that?

Academia, bureaucracy, analytics, statistics, and democracy are all biased against the individual in favor of the collective, incremental good. But does collective, incremental improvement create happiness for the individual?

Are collective, incremental gains concurrent with the goal of education or is education a more personal, individual mission? In education, are we filling of the bucket or are we defining the bucket? Is it possible that conformity and standardization are extinguishing the flame of motivation as a byproduct?

This bias towards systematizing our schools is inherent in the modernization of the nation’s education. Individual differences are smoothed out by numbers and the students’ “superfluous,” individualized interests are left superfluous. The opportunities to create inspiration and ignite motivation are rarely measurable. What is the metric for inspiration?

Is this a defeat of spirit?

In psychometrics classes, large numbers and extensive sampling are necessary. Individuals are carefully conceptualized before all tests are administered, but sometimes the validity of the test isn’t well-known until the test has already been taken. Education gap wideningIn a normal high school history class, tests are content-driven and highly specific inquiries about the contents of your bucket. While in a job interview, the applicant is supposed to “stand out” and be a “special” graduate of the content-specific combine, i.e. their college and major.

Troubleshooting this dichotomy is a losing effort. These two processes of ordering achievement negate one another. Teachers are being forced into administering the contents that are allowed into the bucket, students are having the bucket poured over their heads, and the testing community is salivating over the NCLB-allocated government contracts.

The goal of hermeneutics is a humanist, idiosyncratic interpretation of one student while psychometrics assigns a model to our cognitive processes. Psychometrics includes “measurement” and cognitive “diagnosis” schemes that assert models of the average examinee’s content-specific thought processes. Human functions are implicitly algorithmic here, and the randomness and blurriness of higher-order processes, in effect the “humanity” of the individual, has been stripped because it can not be modeled with graphical analysis and Bayesian estimation. Variance cannot be eliminated from the human interactions that culminate in gaining an “education.” Algorithms cannot replace the teacher-student relationship.

Capitalism is an economic philosophy that lauds individual gains as a function of efficient supply chains, automation, and forecasting, i.e. systematization that streamlines productivity and profit for the few with the power to implement it. If we aren’t careful, the success of private equity hedging may influence the application of this perspective onto the nation’s education system.

Despite the fact that the natural order human beings evolved within, the current procedures for creating predictable results subverts spontaneity and subjugates nature into something repeatable and essentially unnatural. In contrast, babies emerge from the womb with eager eyes… learning to eat, crawl, walk, speak, run, jump, emote, share, etc. This natural urge for learning is then being confined to a classroom that is becoming more standardized… and to the child, it is rendered flavorless.

The problem is that a human being, whether you’re talking about the student or the teacher, will not go gently into that pre-packaged destiny. We must determine how to ignite their curiosity, give flavor to the content, learn from the data, and resist the temptation to eliminate the variance of the model.

Read More

The NEW mathematical tools of Education Administration

The NEW mathematical tools of Education Administration

Research in education is finally approaching the next level, and judging by my graduate classes in statistics, we’re going to need some incredible brain power to use these new methodologies.

First and foremost on the VAM (Value-Added Modeling) for evaluating teachers, schools, and districts, the premiere tool is called HLM (Hierarchical Linear Modeling). To use this software reasonably well, you need about two courses in basic statistics, a course of linear regression, and another course of multi-level modeling. It’s a pain in the arse to interpret the numbers that come out of these models, but if you can do it, you can probably work for anyone from Operations Research departments to Goldman Sachs. As a quick summary, if you input last year’s scores, students’ socioeconomic status, gender, race, parents’ education, etc. then you should be able to predict a student’s score the following year. In companies if you plot the productivity vs. sick days used, chances are you’ll see a relationship in the data. If you’re working through multi-level regression work, I’ve found these resources that carried me through a few homework assignments as well as a midterm:

Multilevel Regression Modeling Resources

Probability has also become a major focal point for estimating how random characteristics of schools, teachers, and students can be modeled. For a great introduction to probability theory, Ross is the major book that all beginners seem to find the most appropriate:A First Course in Probability (8th Edition). The major takeaway from probability theory is the “probability distribution.” For the best possible overview of the interconnectedness of this (insanely) difficult topic (well, at first…), this paper is a beautiful graphical organizer. Anyone who begins this little tumble down the rabbit hole will also be drawn to Bayesian Data Analysis. You’ll also probably need to get a good grip on R (the programming language) as well… So, if this is you and you’re trying to figure out this mess of Bayesian analysis (also related to Artificial Intelligence, Neural Networks, and forecasting), you’ll probably want this book with the cute little puppies on the front:

Finally, IRT (also known as Item Response Theory) is how they grade the SATs. CDM (Cognitive Diagnosis Modeling) is an evolution of IRT and is the umbrella concept of the multifaceted models that have arisen to explain the ways we can test for specific skills. It is extremely complex and requires algorithms such as the Expectation-Maximization Method or Maximum Likelihook methods (which again, require insane amounts of statistics training in probability and inference).

If you’re looking into developing a strong program in statistical analysis in education, this is your foundation. Policy and management aside, this is how to extract the data that is required to make policy arguments. If your superintendent isn’t aware of these tools, it’s time to go to a board meeting.

Read More

Mid-Semester Musings…

Mid-Semester Musings…

Hurricane Sandy is, ironically, providing a short respite… the power is still on, classes are cancelled for two days, I have a moment to make updates on the sites I’m administering, and I can finally reorganize my winter clothes… well, while there’s still power and light…

Since September, I’ve learned and accomplished far more than I expected. For starters, statistical mathematics is much harder than I thought… but much more powerful than I imagined. Probability is a way of envisioning the separation of one potential event into all of its possible actualities, and the defining attributes of the players in this universe of events are definable “after the fact.” It’s actually quite incredible. I now understand why financial analysts think they’re smarter than everyone else, but I still can’t fathom the arrogance that usurps their morality. But to be honest, the difficulty of using these statistical algorithms to make good predictions are far more valuable than any public sector salary… so any true nationalist needs to do some introspection about their tax investment if they want smart people to assist in the reformation of this country. It’s no wonder that analysts would rather make 6 times more money for easier work. (Easier you ask? Yes… because statistical research for social science and psychology is still developing out of its infancy, but the research for finance is 45 years ahead.)

The election is almost over, thank heavens. I’m, admittedly, a politically vested person. I believe in the common good, the general welfare, prevention of warfare, promotion of education and healthcare… and it’s very sad that we can’t be sure that either candidate will push these efforts. What’s even worse is the insane amount of money that runs the PACs that support these two political parties that have a duopoly on our political system… and it seems that our population is far too immature and short-sighted, ignoring the tremendous oligarchical structure that holds our pensions, municipal bonds, college funding, mortgage banking, and our news organizations. Most believe third party candidates to be impractically naive personalities. They may actually be our only way to reform a broken government.

On a personal note, the edits for our book on Model UN Education and Social Intelligence for high school students is complete and finished. The website is ready for media content and our videos from the upcoming RUMUN and HMUN workshops will be available to members, as well as my upcoming presentation on “Presentation Skills: An Application of Social Intelligence” that I’ll be giving to the Rutgers Education Psychology faculty and graduate students. The youtube channel is up and running, the copyright is initiated, and the affiliates are being notified of their ability to make commissions. If you want to help sell our book for online commissions, let me know!

My wife is also beginning her work on her little website adventure. She will be launching her site called the Uneasy Yogini within the next two months. She’ll be promoting her classes, good products, healthy food and exercise, and some nice local spots. One day she’s hoping to have her second career take over her primary… but that is several years away, and we’ve got some parental basics to get through first.

 

Read More

Transitions…

Transitions…

This day is usually filled with brooding…

Strolling back into a high school after a 2 month vacation and seeing your coworkers is a happy occasion that gets quickly thrown to the side as socially ardent teenagers quickly smash through the doors and shout to their supporting cast of boisterous pseudo-hooligans while we make our opening speeches. Teaching is honorable… and our abilities of crowd control are rarely acknowledged.

Today, for me, began with a text from my coworkers… cursing me from their annual re-insertion into public sector protocol while I lay in bed thinking about how best to mentally prepare for my 6:30-9:30pm graduate class on Probability. My daily stress level during the “school year” has officially plummeted from an 60/100 to a 20/100. My advisor has even shunned my overzealous work ethic and told me to “settle in” to my schedule. Wanna know my schedule that needs settling into? M 4:30-7:30pm, T 6:30-9:30pm, W 4:30-7:30pm, & Th 4:30-7:30pm. I think I’ll hit the gym during the days that I work on my comfort level.

Now, it’s not all easy. Specifically, I have very little experience with this new form of statistical mathematical notation. The language of math of physics is much more visual and robust. We don’t randomly insert a vector term unless there is a physical property that includes direction… whereas, statisticians can randomly insert and fit an extra nominal term within a model that has no physical interpretation; it’s just data. It is data that may or may not be important. Rarely do physicists do anything “extra.” We are barebones, brute force mathematicians that learn the elegant tricks when we have to. So my prediction is that this change in philosophy is going to carry my forehead into the figurative brick wall as I glean the nuances of Item Response Theory and Hierarchical Multilevel Analysis.

On another note, my fourth class in “Teacher Evaluation” is going to require a great deal of empathy. Subjective/Qualitative Evaluation is farthest thing I’ve ever witnessed from a hard science that attempts to portray itself as an evolved, structured analysis. I have a feeling that I’ll be making hand waving arguments and laughing at the false pretenses of administrators. Nearly every paper on teacher evaluation that I have ever seen, shows that local administrators show extreme bias in their evaluations… so much so that third-party university professionals are usually called in to “correct” for the skew in qualitative data. Sadly, qualitative data is still the only evaluation method my former colleagues and I have ever been subjected to.

 

Read More