Author Topic: NEW! Galaxy Zoo science blog!  (Read 115435 times)

garrett_cw

  • Hero Member
  • *****
  • Posts: 2805
  • In this picture I'm the Grandfather.
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #15 on: January 02, 2008, 04:25:25 pm »
I have started a new topic

Galaxy Zoo Forum > The outer Universe > Cafe at the end of the Universe (Moderators: Alice, Edd, StuartA) > FORTRAN, Computers, Computing and the like

for a discussion of the stated topic. In what I think is the proper place.


Moderators are welcome to correct me without thought of offending me.

Curtis W. Garrett

Curtis
"To get along in life you must be very clever or very nice. I've been clever. I recomend nice."

zookeeperKevin

  • Galaxy Zoo Keeper
  • Admin
  • Hero Member
  • *****
  • Posts: 2020
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #16 on: January 02, 2008, 05:55:33 pm »
Thanks, garrett, that'll keep this thread uncluttered.  :)

Alice

  • Global Moderator
  • Hero Member
  • ****
  • Posts: 31782
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #17 on: January 02, 2008, 05:57:00 pm »
Here's the topic.  :)

EigenState

  • Guest
Re: NEW! Galaxy Zoo science blog!
« Reply #18 on: January 02, 2008, 06:57:22 pm »
Greetings,

Quote from: Zookeeper Anze “Final pre-bias data download” on the blog
In the reweighted sample, we try to improve the sample by essentially comparing the agreement between users: the idea is that if ten users claim that a certain galaxy is a spiral and the eleventh users says it is an elliptical, it is likely that the 11th user got it wrong. Users who commonly disagree with everyone else gets down-weighted and those who always agree get up-weighted.

It is a purely statistical exercise meant to remove pranksters that click randomly and up-weight careful users. In practice, we can check how well it works. We do this (well, Steven does it) by looking at galaxies that have the same absolute luminosity and size and shouldn’t evolve over the small redshift range probed by the SDSS. The upshot is that it doesn’t work as well as initially anticipated…

Notwithstanding that the reweighting “doesn’t work as well as initially anticipated”, might the results of this statistical analysis be of value to the participating membership?

Consider adding the weighted performance of each individual member to that member’s “Show My Galaxies” page, along with some measure that places their individual performance in context with respect to the performance of the overall membership.  In that way the information remains private, yet provides an important opportunity for individual members to evaluate their performance with the sole objective being to identify potential weaknesses in their classifications and, with that knowledge, undertake to strengthen their classification skills.

Ideally, weighted performances as a function of morphological category would be most valuable.

This is nothing more than a simple feedback mechanism that allows us, the participants to improve the objectivity of our contributions.

Best regards,
EigenState

zookeeper_Kate

  • Full Member
  • ***
  • Posts: 140
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #19 on: January 02, 2008, 09:07:32 pm »
This is nothing more than a simple feedback mechanism that allows us, the participants to improve the objectivity of our contributions.
Hi Eigenstate, thanks for you comments.
Indeed we did think about this for a bit, but alas I reckon it's a bad idea to publish users weights for a number of reasons - but mainly because it is very unclear what the weights really say about a user. For example, a user might be really upset to see that they have a low-weighting, indicating that they often disagree with the majority of people (if we use a democratic weighting scheme). They might then try to alter their classifying behaviour so to 'improve' their weighting.  This would be disastrous, because perhaps they are actually excellent at classifying, and much more meticulous than the majority of other users. In this case a low weighting would be good! The majority ain't always right, right?!

If providing this kind of feedback was to have any effect then this would be bad - because it would mean our data becomes correlated in complicated ways that we can't trace i.e. the results from one week can effect the next week. Ultimately everyone will want to up their weighting, and I can imagine a horrible situation where everyone just clicks elliptical all the time! Cause this would give everyone great weightings - but completely ruin the project!

Therefore, you see that knowing your weight can only have a bad effect (if it isn't going to have an effect then there's no need to know  ;) ). Ideally we don't want anything to have an effect on you - we want everything to be as unbiased and open and transparent as possible when it comes to analysing the results.

Plus the weights change all the time, as more classifications are made and it is computationally very intensive to compute them. Further, there's an infinite number of ways of working out the weightings! We could see how well you agree with each other for just the bright galaxies, or how well you agree with an 'expert'. But the point of all this is that we do not know the true morphology of these galaxies - and therefore we cannot give you a true weight (ie. how well you are classifying).

I hope that helps to explain our situation a bit! I appreciate that it must be a bit frustrating not to get more feedback on your classifications. Perhaps when this phase of the project is wrapped up then we can feedback more... and ultimately all this data is probably going to be made public!
Cheers, Kate
« Last Edit: January 02, 2008, 09:35:08 pm by zookeeper_Kate »

Hanny

  • Global Moderator
  • Hero Member
  • ****
  • Posts: 17256
  • "Voorwerp kid"
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #20 on: January 02, 2008, 09:19:44 pm »
Thanks Kate, great explanation, I don't want to know ;D ;)

EigenState

  • Guest
Re: NEW! Galaxy Zoo science blog!
« Reply #21 on: January 02, 2008, 09:43:10 pm »
Greetings Kate,

Understood.

Does this mean that I can invoke similar reasoning and stop writing examinations as well?   ;)

Best regards,
EigenState

Alice

  • Global Moderator
  • Hero Member
  • ****
  • Posts: 31782
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #22 on: January 02, 2008, 09:45:21 pm »
I know you're only joking, EigenState, but I do want to point out one thing - Galaxy Zoo isn't an exam. The galaxies aren't already classified in advance of us, and our classifying isn't testing us. It's us generating the data. The only thing we really have to measure our own results against is what other people think, and there's nothing to say the "other people" are necessarily correct.

Sorry if that was obvious . . .

Half65

  • Hero Member
  • *****
  • Posts: 10288
  • "Eagle Nebula"
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #23 on: January 02, 2008, 10:52:05 pm »
I totally agree with Hanny and Alice.
This zoo its a challenge but in a scientific contest and I think that it's not the scope of the research.
Sorry if I'm not succeed to explain the concept in English.  ;)

mushroom

  • Hero Member
  • *****
  • Posts: 524
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #24 on: January 03, 2008, 05:09:29 pm »
Certainty is just plain tricky isn't it? Like a ruler trying to measure itself - not easy.  ;)
« Last Edit: January 03, 2008, 05:11:53 pm by mushroom »

garrett_cw

  • Hero Member
  • *****
  • Posts: 2805
  • In this picture I'm the Grandfather.
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #25 on: January 03, 2008, 10:47:24 pm »
there really are two places where this could go.

People want to get better at the classifying.
This is good and should be facilitated.
The people doing the study are concerned that
giving numbers back to the participants
about how well they agreed with the other participants will introduce
bias in complicated ways that will be difficult or impossible to
account for or possibly even detect.

If the data is looked at from a different angle then perhaps both needs
can be satisfied.

In stead of how well did a particular person do let us ask
how difficult was it for people to aggree and why?

For a particular image, if 99+% agree on the classification
then there is no problem to improve on here.

On the other hand if the answers or distibuted 33%, 33%, 33%, 1%
for CW, ACW, EDGE/DK, and eliptical, there could be an opportunity
for skill improvement. WOLOG  ;D if we assume the 1% for eliptical is a
result of "damn I hit the wrong button." the other number lead to
some interesting questions.

Then looking at the image---
What caused a third to think this went one way and a third the other?
Was the Edge/DK from the bias part of the study where something had been done to the image?
Was the EDGE/DK possibly the best answer and the other two thirds the result of
people streching to put the image in either CW or ACW?

The same or similar questions couldbe asked for an 80/20 split.

From this a Second Tutorial could be written for people to use to sharpen their skills.
I see this as an additional one not a replacement.

Curtis
Curtis
"To get along in life you must be very clever or very nice. I've been clever. I recomend nice."

EricFDiaz

  • Guest
Re: NEW! Galaxy Zoo science blog!
« Reply #26 on: January 04, 2008, 12:09:17 pm »
This would be disastrous, because perhaps they are actually excellent at classifying, and much more meticulous than the majority of other users. In this case a low weighting would be good! The majority ain't always right, right?!

I agree with Kate. I'm frequently in disagreement with the majority, and I'm usually right! ;D ;D ;D ;D ;D
Sorry, I couldn't resist. :D ;)

Hanny

  • Global Moderator
  • Hero Member
  • ****
  • Posts: 17256
  • "Voorwerp kid"
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #27 on: January 04, 2008, 12:31:27 pm »
This would be disastrous, because perhaps they are actually excellent at classifying, and much more meticulous than the majority of other users. In this case a low weighting would be good! The majority ain't always right, right?!
I agree with Kate. I'm frequently in disagreement with the majority, and I'm usually right! ;D ;D ;D ;D ;D
Sorry, I couldn't resist. :D ;)

lol :D

Mark OConnell

  • Sr. Member
  • ****
  • Posts: 418
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #28 on: January 04, 2008, 03:05:33 pm »
If they want to get better and be compared to others why not a follow on test. We all get tested anways just to participate. Why not make it a heading of "For Those Who Like Tests and Getting Compared With Others Click Here." The rest of you can go start classifying.

 ;)

zookeeperJordan

  • Admin
  • Jr. Member
  • *****
  • Posts: 78
    • View Profile
Re: NEW! Galaxy Zoo science blog!
« Reply #29 on: January 07, 2008, 05:35:14 pm »
Hey all,

New post up today!

Kevin describes his research into "blue ellipticals" - elliptical galaxies that, contrary to our expectations, are forming lots of new stars. Have a look at the blog:

www.galaxyzooblog.org