There Is No Right and Wrong

Continuing this series of posts, answering questions from Milky Way Talk, MWP user UncleClover writes:

I’m a very fast learner, but only if there’s some sort of feedback going on letting me know what I’m doing right and what I’m doing wrong. Once an image that we notated has been processed by a professional, is there any way for us to look it up and find out how accurate or inaccurate our notations were?

Feedback wouldn’t even need to necessarily be an extra step, just having access to your own prior notations and being able to compare them to the “final conclusions” of the professional analysis would be useful.

And in a similar vein, user katieofoz writes:

I was just wondering about a few thing that someone here might be able to answer.

• Im assuming the data collected for each image is compiled and if a similar marking shows up a certain number of times someone (with actual qualifications) looks at the image and classifies it? Is this right?
• From the data collected is there anything that looks at how often a person is correct in their markings? Like some sort of way of ranking the user based on their observations?
• What is done with the compiled data afterwards? Is it based off the average of observations or are areas looked at by qualified individuals to map it more accurately?

An important thing I learnt during my PhD is that there’s an awful lot of work involved in research that you really don’t need a PhD for. Or even a degree. Just a pair of eyes will do, and a few fingers. The tasks you’re performing for Milky Way Project are a great example of that. I may well have a PhD, but I’m no better than any of our users at finding or drawing bubbles. So really, we’re all experts in this project, and as long as you’re genuinely drawing what you believe is a bubble, or a knotty thing, or a dark cloud, there is no “right” and “wrong”.

To be clear, all we’re doing in the science team is providing you with the dataset, and then gathering up all the drawings of all the users and merging all that information into a consistent catalog of objects. We implement a few quality control measures, for example we consider a new user’s first few drawings to be practice drawings and don’t enter those into our dataset – as UncleClover says, there is a learning process.

We make no judgment on whether a drawing is “right” or “wrong”, and we don’t process individual images or users’ drawings. Once you’ve had a little practice, every click counts. All the clicks go into a (giant!) database, and a computer program written by Robert Simpson analyses all the data.

This program essentially scans the whole are of the sky covered by our images and locates clusters of ellipses that users have drawn (you can see an example image with all the various drawings made my volunteers, above). Where we find more than a given number in a small region, we mark this as a bubble. The bubble’s properties that go into the catalog are calculated from averages of the individual users’ classifications – as katieofoz rightly suggests. So whether a bubble you drew ends up in the catalog really just depends on whether lots of other people agreed with you. Given that there’s no real “right” or “wrong” in MWP, we don’t rank the users. It’s not a competition! But our processing algorithm does allow the classifications by experienced users to count more heavily than those of newbies.

Rob wrote more about the procedure earlier on this blog, and of course this will all be described in a lot of detail in the forthcoming paper.

If you haven’t done so already and you’re interested in knowing more about the infrared bubbles in the interstellar medium, I’d invite you to read the original bubbles papers of 2006 and 2007, written by our science team members Ed Churchwell (U Wisconsin-Madison), Matt Povich (Penn State), Bob Benjamin (U Wisconsin-Whitewater), Barbara Whitney (U Wisconsin/Space Science Institute) and their collaborators. This paper is packed with information on the Spitzer surveys we’ve taken our data from,the difficulties in picking out bubbles and what the bubbles might physically represent. The references are below, and the paper should be available via the ADS link.

Churchwell, E., Povich, M., Allen, D., Taylor, M., Meade, M., Babler, B., Indebetouw, R., Watson, C., Whitney, B., Wolfire, M., Bania, T., Benjamin, R., Clemens, D., Cohen, M., Cyganowski, C., Jackson, J., Kobulnicky, H., Mathis, J., Mercer, E., Stolovy, S., Uzpen, B., Watson, D., & Wolff, M. (2006). The Bubbling Galactic Disk The Astrophysical Journal, 649 (2), 759-778 DOI: 10.1086/507015 [ADS]

Churchwell, E., Watson, D., Povich, M., Taylor, M., Babler, B., Meade, M., Benjamin, R., Indebetouw, R., & Whitney, B. (2007). The Bubbling Galactic Disk. II. The Inner 20o The Astrophysical Journal, 670 (1), 428-441 DOI: 10.1086/521646 [ADS]

Zooming and Drawing

Continuing this series of posts, answering questions from Milky Way Talk, here’s one from MWP user broomrider1970, who asked about a zoom option for the images. In fact there’s a whole thread about this on Talk.

When we first started planning the Milky Way Project (MWP) and began testing the interface, we actually had quite a lengthy discussion about a zoom option. In fact, digging through my email, it was my very first question to the developers. As it turns out, adding the zoom option is first of all quite challenging technically. Secondly, and more importantly, giving users the ability to zoom in on images gives us an extra level of uncertainty when we’re trying to process the classifications. By keeping the image static we know for sure that all the users saw the same image in the same way, and we know exactly what the minimum and maximum bubbles sizes are for each image. So it was really an issue of ensuring consistency in the classifications.

Next up: user chairstar asked:

Since I’ve found so many areas of interest – not just IDRC’s but other things like green knots, which do not have a round or square shape, I’d like to see if there is a way of allowing us to ACTUALLY drawing “lines” around irregularly shaped areas, as opposed to being constrained to the classic square and round shape that we are to use now.

Perhaps incorporating a drawing software, allowing free-hand lines could be put in.

Again, this is something we thought about. Particularly for the infrared dark clouds (IRDCs), which tend to have complex filamentary shapes, we wanted to have some kind of polygon- or freehand-drawing tool. But as with the zoom option, having freehand drawings as classifications makes it very challenging for us to merge all these drawings into a consistent catalog of objects. I’m also not sure how we would codify the information captured in freehand drawings in an easily accessible format.

In addition, for the specific case of IRDCs, it wasn’t clear that this would give us better results than an automatic detection algorithm. Several existing IRDC catalogs detected dark clouds with algorithms, and these actually do quite a good job. Not wasting people’s time on tasks that are done just as well by a computer is a core principle of the Zooniverse’s citizen science philosophy.

Bok Globules and Herbig Haro Objects

The Milky Way Project science team are currently busy laying what we hope is the final hand on our first publication. In this paper, we’ll describe the project and why we decided to take the citizen science approach for the task of identifying bubble structures in the Galaxy. We will also present our first results from the hundreds of thousands of classifications we’ve logged on the site, and how our new bubble catalog might be useful for further studies of star formation and the interstellar medium. As we’re big fans of open data sharing, the paper will of course be made publicly available via Arxiv.

I spotted a bunch of interesting questions on the Milky Way Project Talk forums recently and wanted to take some time to jot down a few answers. Here the first one (or in fact, two).

User Ken Koester asks:

1. Is the resolution of these images such that we ought to be able to detect Herbig Haro objects?
2. Bok globules are pretty cold; do they still show as black in these images?

Herbig-Haro objects were first discovered in the early 1950s by, not surprisingly, astronomers George Herbig and Guillermo Haro, who spotted optical nebulosity in active star forming regions like Orion. Decades of further observations in the optical, infrared and radio have since established that these “nebulae” are bright knotty streams or jets, sometimes with a very marked bipolar and narrow shape, streaming out from newly forming stars. This type of powerful outflow of material is a typical feature of star formation at all different masses.

The optical radiation seen from these Herbig-Haro flows arises when material in the outflow powers into the quiescent medium surrounding the new star, causing shocks in the gas. Such shocks are also commonly seen in the infrared. In the Spitzer bands of observation, shocks are particularly prominent in the 4.5 μm channel – the blue channel in our Milky Way Project images. Outflows come in variety of size, and of course their apparent size depends on how far away they are. But the resolution of Spitzer is certainly sufficiently high to spot them.

If you want to know more about these objects, there are two excellent review articles from Annual Reviews of Astronomy & Astrophysics on the topic:

Schwartz, R. (1983). Herbig-Haro Objects Annual Review of Astronomy and Astrophysics, 21 (1), 209-237 DOI: 10.1146/annurev.aa.21.090183.001233 [ADS]

Reipurth, B., & Bally, J. (2001). Herbig-Haro Flows: Probes of Early Stellar Evolution, Annual Review of Astronomy and Astrophysics, 39 (1), 403-455 DOI: 10.1146/annurev.astro.39.1.403 [ADS]

Bok globules are very dense and compact cloudlets that are forming new stars in their interiors. They appear black in optical and infrared images, as they’re too cold to emit any radiation shortward of around 100 μm. To study what’s happening on the inside, we have to observe at those wavelengths and beyond, which are covered by e.g. the Herschel Space Observatory and millimeter telescopes such as IRAM or ALMA. They should be visible in our images, as they appear dark in all our colour channels. But not many are as distinct-looking as the ones we see in beautiful Hubble images like this one, so they can be very hard to spot.

The Spitzer image below of a giant Herbig-Haro flow inside a Bok globule towards the constellation of Vela, at a distance of 350 pc (1140 lightyrs), combines light at 3.6 µm (blue), 4.5 and 5.8 µm (green) and 8.0 µm (red). The colours used are a little different than in our Milky Way Project images, which use 4.5/8/24 µm respectively. The image below looks at an area on the sky of 10.2 x 6.5 arcminutes, which is just slightly closer in than the highest zoom level of the MWP images (18 x 9 arcmin). Compared to other H-H objects, HH46/47 is really enormous, so other outflows are likely to appear much smaller.

Spitzer’s view of a giant Herbig Haro flow, HH46/47, inside a Bok globule (Image: NASA/JPL-Caltech/A. Noriega-Crespo (SSC/Caltech), Digital Sky Survey).