Rubrics for Scientific Interaction

Including: [lab notebook] [short talk] [scientific report] [nanoworld webquest] [peer review] [test essay] [discussion fora] [one-liner]

The first three of these rubrics were developed for use with experiments, each lasting one-third of a semester, done by students in the Advanced Physics Laboratory (Physics 311) class at UM-StL. Each experiment resulted in one lab notebook, one final report, and two talks (one halfway through) reviewed by its audience members including student peers. The fourth rubric is under development here as a tool for use with short classroom/web nano-explorations designed to enhance progress toward the discovery and reporting goals of present day introductory science courses. The fifth rubric is under development for evaluation of student peer reviews. The sixth describes a compact way to provide feedback to students on exam essay-question answers. All of these are based loosely on the format used in modern industry for goals setting and results reviews, except that for industry applications the sequence BASIC, GOOD and EXCELLENT may be used in place of BASIC, COOL, and FAR OUT./pf

Sample Lab Notebook Rubric

This is a daily diary of the progress on the experiment. For our advanced physics lab course, it will be turned in at the end of the experiment to be graded. A new notebook for each experiment, or perhaps better a single bound notebook with pre-numbered pages, should be used. The notebook should contain detailed descriptions of what you are doing, and should also pose questions to be answered, make predictions, and contain a dialog with yourself. In other words, make the notebook an integral part of the experimentation process. As you work and make notebook entries, record the date and time. Write clearly, and make drawings whenever possible. Use pen and do not erase mistakes, but line them out neatly.

A forty-point notebook rubric might look like...

Notes about the patent perspective on documenting scientific discovery can be found here.

Sample Short Talk Rubric

For our advanced physics lab course, these should be about six minutes long and include the following:

a). Tell the audience what you are going to say, say it, then summarize at the end.

b). Use an overhead projector with transparencies, or presentation software, as these are the most generally available ways of presenting material. Transparencies can be made with copier, computer printer, or simply by hand.

c). Choose topics that will be informative and interesting, and related to the experiment at hand. However don't choose so wide a subject that you cannot keep the talk within the 6 minute target. Stick with that chosen topic.

d). Following each talk, be prepared to answer questions and participate in a discussion about what you have presented. Those in the audience will be expected to do the same, and possibly to help in providing some constructive evaluation of the presentations as well.

A talk rubric for peer evaluation, using a forty-point scale, might look like...

Sample Scientific Report Rubric

When used for our advanced physics laboratory course, these are due 1 week after the experiment is completed. Is this also a reasonable rubric for possible submissions to the LosAlamos/Cornell eprint arXiv? How might we make it even more appropriate for that purpose? Getting back to guidelines for our advanced lab:

a). Be terse and only a few pages in length.

b). Include enough information that the experiment can be repeated by an independent student observer who is at a junior physics major level.

c). Use a simple format - Abstract, Introduction, Experimental Details, Your Observations, Interpretation/Theory and Discussion/Summary.

d). The American Institute of Physics "Style Manual" and/or a sample from Physical Review can be used as a guide.

e). Include in the paper some reference to contemporary research (less than 10 years old) related to the experiment, if you can locate such references.

A forty-point report rubric might look like...

Sample nanoWorld webQuest Rubric

This is a scientific diary and report of one of your adventures as a nano-detective. We recommend that it have six sections, in the following order:

1). Work Log: This is a simple diary of project activity with entries of the form "date-time/what happened".

2). Introduction: This section describes what you can find out about the challenge before actually doing the experiment. Here references to external sources are crucial.

3). Method: Describe here how you took data, with enough detail that another student in the course could repeat your experiment and get approximately the same results.

4). Data/Observations: This section reports the actual measurements that you made. These might be typed directly into the text, or simply summarized in the text and provided separately in spreadsheet or image attachments.

5). Interpretation/Theory: In this section, you make the connection between your observations and the objective of the nanoworld challenge. What conclusions are possible, based on your observations?

6). Assumptions/Uncertainties: Here is where you address the strength of your interpretation. What assumptions are involved, and how likely are they to be valid? If you have quantitative conclusions, provide evidence concerning the expected numerical error in those conclusions. This might also be a good place to mention possible future work.

A generic nanoWorld webQuest rubric, using a forty-point scale, might look like...

Here are: a web page with some links to nanoWorld webQuests, a draft nanoexercise for introchem, the summary of a proposal about their possible applications, and a review form for fleas on fleas.

Sample Peer Review Rubric

Note: This rubric evaluates the reviewer, not the manuscript submitted. In that context, the most important part of a peer review are the reviewer's suggestions for ways to improve in the "essay portion" of the review. In general, the more (i) homework you do in the review process, (ii) time spent learning about the subject/literature referenced, and (iii) suggestions offered the better even if your feedback at first glance seems annoying to the author. Your suggestions might best be conveyed as items referenced to specific sentences, pages, and/or parts of the presentation. In the case of submitted manuscripts, they also might be conveyed as "markup" or marginal notes.

A final step in a peer review is "filling out the form". This usually includes making recommendations to the editor, as well as some sort of ranking of various aspects of the paper (as for instance occurs in the presentation rubrics above). A sample form for the non-essay component of a peer review, used by a scientific journal, may be found here. This step addresses issues of importance to the editor, but is generally of limited use to the author.

A generic peer-review rubric, which uses a forty-point scale and assumes that the reviewed-report itself is evaluated according to an 8-element rubric like those above, might look like...

Here the goal is to provide structured feedback to students, e.g. on their input to a 5-point essay question, with a string of 4 numbers whose average is the point-score for the problem. We do it here in terms of the "four C's": Completeness, Correctness, Clarity, and Creativity. Thus a grade written as: 4352 → 3.5 means that, for this person's answer, Completeness was 4 (most underlying issues addressed), Correctness was 3 (answers weren't too far off), Clarity was 5 (quite unambiguous), but Creativity was 2 (same old same old). This is a compact way to convey perceived strengths and weaknesses, even if the grading process for such questions is inherently subjective.

A generic test essay rubric, for use with a 5-point question as described above, might look like...

Weekly Discussion Board Activity

Note: This rubric is designed to evaluate participation e.g. in weekly topic discussion fora as part of "one room schoolhouse" courses like the web/lab course at UM-StL on "emergent nanoscience practicals". Like the essay rubric, a subset of the issues evaluated are subjective but can nonetheless be characterized. In addition, however, web discussion packages generally assist with some numerical measures of participation as well. Some of these are grouped at the beginning in the table below.

A generic forum-participation rubric, which uses a forty-point scale and assumes that the reviewed-report itself is evaluated according to an 8-element rubric like those above, might look like...

One-line Quantitative Puzzler

Here, we're working on a rubric to evaluate one-line quantitative problems proposed for use in a course with few math pre-requisites. We'd like these one-liners to be (a) relevant to and do-able with the course material that it targets, (b) useful and/or interesting to those taking the class, (c) understandable, and (d) original. Suggestions for this rubric are invited. As with the test essay rubric above, taking the average of numbers for the four items below puts the evaluation on a 5-point scale. For example, a point assignment of "4353" for the items below gives the problem a score of (4+3+5+3)/4 = 3.75 out of 5.

Quantitative one-liners for various sections of our how things work class might, for example, include "What voltage between you and a doorknob is implied by a [d] cm spark?", "How deep is a hole that takes a coin dropped on earth [t] seconds to hit bottom?", "How many 'gees' of centripetal acceleration does it take to keep you moving in a circular path of radius [r] meters at a speed of [v] meters per second?", and "By what factor does the heat that an object radiates change when that object's absolute temperature changes by a factor of [f]?".