[phil_20686]

 billw55, on 2012-September-14, 06:49, said:

So, if you don't swipe card and sign, what *do* you do? Many stores here no longer require a sig below a certain amount, but no place I know of has dropped it entirely.

You use a pin. More secure, faster, and less open to fraud.

[phil_20686]

 billw55, on 2012-September-14, 06:49, said:

Yes, the difference in the regulatory environment is we have less. In particular, as I said, we rarely have government setting prices (including at zero).

We do not have government setting prices. But we do have the Office of fair trading, which will intervene if it believes that the industry is engaging in a variety of unethical conducts: Recently:

(1) They fined banks for mis-selling PPI. (This is meant to cover you in case you miss mortgage payments, but it turned out the small print meant was misleading, and it was `sold' to lots of people who did not meet the qualifications in the small print. The OFT not only forced it to honour the contracts with people, based on what the consumers thought they were being sold, they fined the banks for every instance of mis sold PPI.

(2) The have forced energy companies to reform their pricing regime, judging most schemes to be unnecessarily complex for the purpose of preventing people from properly comparing, and so inhibiting fair competition.

(3) They have forced mobile phone companies to stop charging extra for calls to foreign mobiles, except insofar as they can demonstrate materially higher costs. It was ridiculous that it was 45p to send a text message from Germany to France, even though they are both served by the same phone network infra structure. Then it transpired that most mobile phone companies were routing them through the internet, and not even using the expensive phone infrastructure that they claimed as justification. Once upon a time there was limited bandwidth for phone communication between, say, london and paris, but those days are over.

But in this case, it was not a regulatory move, it was purely market forces. Debit cards are cheaper than cash. It is actually a large amount of effort for banks to count cash, and fulfil their obligations regarding replacing old tatty notes. This is why they charge fees for change. Use of debit cards is massively cheaper than either credit cards or cash. (to the tune of about 0.05p per transaction or something).

[phil_20686]

 Cthulhu D, on 2012-September-14, 06:22, said:

No-one has successfully developed metrics that separate good teachers from bad ones in a way that can be usefully administered on a large scale that are not overwhelmed by the socioeconomic background of the students and random noise from sample sizes. Additionally, no-one has managed to successfully show that coaching that improves your scores on standised tests results in improved employment outcomes which is the actual objective.

No obviously not. You cannot do that until after you implement the metrics.

What you are saying, basically, is that the metrics currently developed disagree badly with teachers experience, in terms of rating teachers. Now this can mean ether:
(1) Teachers are very bad at knowing who the good teachers are.
(2) The metrics are very bad.

Or more likely, both. It is already well established that taller people are widely regarded as more successful teachers. If you attach photographs of the candidates to exam papers, it is well known that this massively influences the marking. Even knowing the gender of the candidate will affect marks in unpredictable manners. How can you really believe that, in the case of teachers, principals everywhere are making good hiring decisions?

I would imagine that teaching would be even worse, since people are likely to be selected for being `likeable' and conventional, based on the `rational astrology' (see interfluidity) which is typical for institutions.

When teachers unions realise that they are at least as terrible (and likely even worse) at making judgements about who is a good teacher, then we will be able to have a rational conversation about how to improve things.

[phil_20686]

Check this out, in one of my favourite pieces of socio-economic research ever done, Harvard economics professors, in response to being evaluated partly bystudent questionaires, constructed a longitudinal study, which conclusively proved that those first year lectures with the worst evaluations had the highest positive effects on student learning outcomes.

The key results are (complete study here)

Quote

Results show that there are statistically significant and sizable differences in student achievement across introductory course professors in both contemporaneous and follow‐on course achievement. However, our results indicate that professors who excel at promoting contemporaneous student achievement, on average, harm the subsequent performance of their students in more advanced classes. Academic rank, teaching experience, and terminal degree status of professors are negatively correlated with contemporaneous value‐added but positively correlated with follow on course value‐added. Hence, students of less experienced instructors who do not possess a doctorate perform significantly better in the contemporaneous course but perform worse in the follow‐on related curriculum.<br style="color: rgb(51, 51, 51); font-family: Arial, Helvetica, sans-serif; font-size: small; line-height: 20.78333282470703px; ">
Student evaluations are positively correlated with contemporaneous professor value‐added and negatively correlated with follow‐on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value‐added in follow‐on courses). Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this latter finding draws into question the value and accuracy of this practice.

So even students cannot judge who their good teachers are. lol.

EDIT: changed link to correct paper. My bad.

[billw55]

 phil_20686, on 2012-September-14, 07:45, said:

...
How can you really believe that, in the case of teachers, principals everywhere are making good hiring decisions?

Really agree. Almost every anti-union op-ed piece I see portrays principals as leaders of the light brigade, the saviors of the system, who will do great wonderful deeds if only freed form the shackles of the teachers union. In reality there are just as many bad principals as there are bad teachers (proportionally). They can and do harm both students and teachers with their misguided actions, including hiring/firing, but certainly not limited to that.

 phil_20686, on 2012-September-14, 07:45, said:

When teachers unions realise that they are at least as terrible (and likely even worse) at making judgements about who is a good teacher, then we will be able to have a rational conversation about how to improve things.

I sometimes wonder why the teachers themselves take no interest in identifying the bad ones among themselves, and addressing the problem. For example, police have a union, and yet most good cops dislike bad cops, and want them dealt with. I can hardly believe that good teachers actively want to do nothing about the bad ones, but the union position seems to indicate this. What's up?

[hrothgar]

Couple quick comments:

I think that Phil is hitting the nail right on the head when he focuses on the difficulty of creating accurate metrics.

Unfortunately, while standardized tests are both objective and popular, they are very severely flawed for measuring teacher effectiveness. Standardized test scores are inaccurate and they incentivize the wrong behavior.

Longitudinal studies have shown that the performance of individual teachers varies enormously from year to year. Teacher effectiveness is important, however, ultimately it gets swamped by various environmental factors. As a result, it is completely unfair to use standardized test scores as a primary metric for compensating teachers.

Sadly, the easiest way for teacher's to raise test scores is for them to falsify test scores and teaching students how to "game" the test...

I know that this isn't going to be popular, but the best way to fix the system is probably to throw money at it...

I know that this is going to come as a shock to people, but US teacher's don't get paid very much. Historically, ingrained sexual discrimination allowed the US to pay teachers squat while still attracting highly qualified candidates. These days, women can get good jobs in the broader work force. This is doubtlessly good for society, but bad for teacher quality. (As I understand matters, these days most teachers are drawn form the bottom third of the academic population).

Plain and simple, if you want to start attracting quality teachers you're going to need to

1. Pay them a lot more
2. Accord them a bit more respect in society
3. Give them a lot more autonomy

FWIW, my mother used to taught junior high and high school level German and French for 20 years before she got fed up and moved to teaching college. She forbade me to ever go into public education. (And before you make the obvious jokes, this has nothing to do with my personality, but rather how unpleasant the entire profession has become)

[phil_20686]

 hrothgar, on 2012-September-14, 08:21, said:

Couple quick comments:

I think that Phil is hitting the nail right on the head when he focuses on the difficulty of creating accurate metrics.

Unfortunately, while standardized tests are both objective and popular, they are very severely flawed for measuring teacher effectiveness. Standardized test scores are inaccurate and they incentivize the wrong behavior.

SO yes, but I am still in favour of metrics. Despite the difficulties, I think data>>>people. When you realise how wrong and arbitrary most hiring decisions are, you realise that metrics do not need to be all knowing to be an improvement.

[hrothgar]

 phil_20686, on 2012-September-14, 08:32, said:

SO yes, but I am still in favour of metrics. Despite the difficulties, I think data>>>people. When you realise how wrong and arbitrary most hiring decisions are, you realise that metrics do not need to be all knowing to be an improvement.

Don't get me wrong... I love metrics.
Right now, Akamai is paying me a very nice salary to create new metrics.

If anyone can propose a good set of metrics for measuring teacher effectiveness, I'd be all for it.
I haven't ever seen any, nor do I know how to do so.

[phil_20686]

 hrothgar, on 2012-September-14, 08:38, said:

Don't get me wrong... I love metrics.
Right now, Akamai is paying me a very nice salary to create new metrics.

If anyone can propose a good set of metrics for measuring teacher effectiveness, I'd be all for it.
I haven't ever seen any, nor do I know how to do so.

We will only be able to create good metrics, after we collect lots of data. As I understand it, the unions are opposed to data collection, because they fear that principals will use it to get rid of people. And they say that the metrics are bad.

I agree that standardised test data is probably not the best. We need to collect not only standardised scores, but also routinely record the scores of every piece of work, and cross correlate it with family background and income. If we did that, we could start the hard work of constructing a decent metric.

[hrothgar]

 phil_20686, on 2012-September-14, 08:47, said:

I agree that standardised test data is probably not the best. We need to collect not only standardised scores, but also routinely record the scores of every piece of work, and cross correlate it with family background and income. If we did that, we could start the hard work of constructing a decent metric.

Few problems with this plan

1. Both the US and the EU have rather strict privacy laws which impact the ability to collect and use said information

2. There is a lot of excellent sociological work focusing on how perceptions about student abilities influences the behavior of teachers. (One of the most interesting examples focused on how feminine a woman's name was with achievement in math). Heisenberg's uncertainty principle isn't just for electrons...

Have you seen any of the literature studying the Finnish school systems?

[Bbradley62]

I think you're making this way too complicated, especially for elementary schools. It is reasonable to estimate that each class in a given public elementary school is drawn from the same pool of students. So, start by measuring each teacher's students' progress against the rest of their own school. If your students perform at more than, say, 1.5 standard deviations below the mean of your own school, then you are flagged for scrutiny. If it happens a second consecutive year, when you presumably have a different subset of students within the same pool, you need to find a new job.

[phil_20686]

 hrothgar, on 2012-September-14, 09:17, said:

1. Both the US and the EU have rather strict privacy laws which impact the ability to collect and use said information

2. There is a lot of excellent sociological work focusing on how perceptions about student abilities influences the behavior of teachers. (One of the most interesting examples focused on how feminine a woman's name was with achievement in math). Heisenberg's uncertainty principle isn't just for electrons...

Have you seen any of the literature studying the Finnish school systems?

I have not, except insofar as some studies crop up in the British media.

I have seen the studies about marking, and photographs/gender knowledge.

I do not think number one is a genuine problem. There are relatively weak standards for anonymising. For example, they collect income data on students routinely for the purpose of deciding if you qualify for free school meals. You could certainly collect it at the year group level, and I suspect at the class level would be ok. Obviously the teachers should not find out the results of individual schools, but this is reasonably easy to achieve in practice.

Its already virtually compulsory to fill in parental income on university application forms, for the purpose of deciding what student loan support you are entitled too. You can only avoid it if you are not taking student loans, which pretty much marks you as independently wealthy.

[phil_20686]

 Bbradley62, on 2012-September-14, 09:28, said:

I think you're making this way too complicated, especially for elementary schools. It is reasonable to estimate that each class in a given public elementary school is drawn from the same pool of students. So, start by measuring each teacher's students' progress against the rest of their own school. If your students perform at more than, say, 1.5 standard deviations below the mean of your own school, then you are flagged for scrutiny. If it happens a second consecutive year, when you presumably have a different subset of students within the same pool, you need to find a new job.

The problem with this is that the population of teachers is large. So even if the % is small, you are talking about a large number of teachers being arbitarily fired. THere are 500,000 teachers in the US, so I think you would have to get a three sigma result, before you could use it for firing/hiring decisions.

Of course, the correct way to use it is to bring people under scrutiny, and if their peers feel them to be under performing, then they fire. Not to mention, many teachers, like many other workers in various stripes, will go through rough patches in their lives that affect the quality of their teaching. Serious illness in their children/spouse, depression, burnout etc. You should be prepared to tolerate two or three years of bad results from good teachers, if they have a good track record in the past, on the expectation that they will improve in the future.

Few people are equally productive through out their lives.

[hrothgar]

 Bbradley62, on 2012-September-14, 09:28, said:

It is reasonable to estimate that each class in a given public elementary school is drawn from the same pool of students. So, start by measuring each teacher's students' progress against the rest of their own school. If your students perform at more than, say, 1.5 standard deviations below the mean of your own school, then you are flagged for scrutiny. If it happens a second consecutive year, when you presumably have a different subset of students within the same pool, you need to find a new job.

Sadly, your notions of "reasonable" don't accord well with reality

According to the data I've seen there is enormous variance in the Year over Year success of individual teachers.
One year some one is on top, the next he is on the bottom.
In all seriousness, you might as well be flipping coins.

This is not to say that individual teachers don't score well on this metric year in and year out.
Flip enough quarter's and one of them is going to turn up heads 20 times in a row.

However, its not fair to use this as an exclusive (or even a primary) metric for evaluating teacher performance.

[Bbradley62]

All such details can (fairly easily) be worked out: career average of ratings, how many sigma, etc. I think the major point is that doing something is better than not, and standardized tests can be used for this purpose if done in a reasonable way. But, I do not think "if their peers feel them to be under performing" is at all relevant; I would expect members of a union to circle the wagons and try to protect each other, even when that protection is undeserved.

[semeai]

 phil_20686, on 2012-September-14, 07:52, said:

Check this out, in one of my favourite pieces of socio-economic research ever done, Harvard economics professors, in response to being evaluated partly bystudent questionaires, constructed a longitudinal study, which conclusively proved that those first year lectures with the worst evaluations had the highest positive effects on student learning outcomes.

The key results are (complete study here)

So even students cannot judge who their good teachers are. lol.

Your quote is not from the paper you link, which furthermore does not have the conclusions you suggest. Furthermore, the authors are from U of Toronto. The closest they come is that perceived easiness of a professor is negatively correlated with student performance in subsequent courses in the same subject area.

The quote is instead from this paper by someone from from UC Davis and someone from the US Air Force Academy, which does have the results you summarize. Namely, student evaluations are positively correlated with student performance in the current course the students are taking, but negatively correlated with student performance in a subsequent course in the same subject area.

No comment on the conclusions of the second paper, I was just thoroughly confused when trying to find the results you suggested in the data from the first paper.

[phil_20686]

 semeai, on 2012-September-14, 10:05, said:

Your quote is not from the paper you link, which furthermore does not have the conclusions you suggest. Furthermore, the authors are from U of Toronto. The closest they come is that perceived easiness of a professor is negatively correlated with student performance in subsequent courses in the same subject area.

The quote is instead from this paper by someone from from UC Davis and someone from the US Air Force Academy, which does have the results you summarize. Namely, student evaluations are positively correlated with student performance in the current course the students are taking, but negatively correlated with student performance in a subsequent course in the same subject area.

No comment on the conclusions of the second paper, I was just thoroughly confused when trying to find the results you suggested in the data from the first paper.

My bad, when searching for the link I had two blog posts open from Greg Mankiw's blog, who originally led me to that paper, I copied the quote from one but the link from the other. It should be the Caroll/west paper "random assignments of students to professors". Will fixx the link in my post.

I remembered the authors as being from Harvard. My bad. Its definitely the caroll/west paper I was thinking of though. Maybe its because Mankiw is from Harvard, and I conflated. Good spot.

[semeai]

 phil_20686, on 2012-September-14, 10:53, said:

My bad, when searching for the link I had two blog posts open from Greg Mankiw's blog, who originally led me to that paper, I copied the quote from one but the link from the other. It should be the Caroll/west paper "random assignments of students to professors". Will fixx the link in my post.

I remembered the authors as being from Harvard. My bad. Its definitely the caroll/west paper I was thinking of though. Maybe its because Mankiw is from Harvard, and I conflated. Good spot.

Thanks. I didn't mean to come off so harsh. The study you meant to link looks well done and the conclusions are fairly stark. They're from data at the Air Force Academy, where they were able to randomize which professor each student got, ensure that the students took follow-on courses, and had consistent grading.

The same sorts of things (and many others) are tracked in the paper you originally linked, at some other university. There the conclusions are all pretty mild it seems. None of the effects of perceived qualities of the professor (perceived by the students) on performance in subsequent courses were significant except for perceived easiness, which was negatively correlated. (They tracked perceived hotness as well, among others!)

I'm not sure what all this means. Could we look at teacher performance in terms of how the students do in subsequent years? This sounds exploitable (campaigning for your previous students to your colleagues), but so does everything else.

[Cthulhu D]

 Bbradley62, on 2012-September-14, 10:03, said:

All such details can (fairly easily) be worked out: career average of ratings, how many sigma, etc. I think the major point is that doing something is better than not, and standardized tests can be used for this purpose if done in a reasonable way. But, I do not think "if their peers feel them to be under performing" is at all relevant; I would expect members of a union to circle the wagons and try to protect each other, even when that protection is undeserved.

No, it's not. The current proposal is a once a year standised test with no socioeconomic controls and fire teachers who don't get good results out of their classes. This is functionally identical to implementing this process: "Do you teach at a poor inner city school? If so, you're fired"

Given that will shortly make it impossible to get people to teach at poor black schools, rather than merely very difficult, obviously that is worse than the status quo. There are a number of reasonable arguments in this thread, but none of them are the argument actually being advanced by the employer.

[mike777]

Just so I understand if you are poor and go to school in Chicago you will never do well on these tests?

You say the only chance is if we tweek the tests for the poor students compared to the middle class students, otherwise the testing will create victims.