Rubrics for Scientific Interaction

Including: [lab notebook] [short talk] [scientific report] [nanoworld webquest] [peer review] [test essay] [discussion fora] [one-liner]

The first three of these rubrics were developed for use with experiments, each lasting one-third of a semester, done by students in the Advanced Physics Laboratory (Physics 311) class at UM-StL. Each experiment resulted in one lab notebook, one final report, and two talks (one halfway through) reviewed by its audience members including student peers. The fourth rubric is under development here as a tool for use with short classroom/web nano-explorations designed to enhance progress toward the discovery and reporting goals of present day introductory science courses. The fifth rubric is under development for evaluation of student peer reviews. The sixth describes a compact way to provide feedback to students on exam essay-question answers. All of these are based loosely on the format used in modern industry for goals setting and results reviews, except that for industry applications the sequence BASIC, GOOD and EXCELLENT may be used in place of BASIC, COOL, and FAR OUT./pf


Sample Lab Notebook Rubric

This is a daily diary of the progress on the experiment. For our advanced physics lab course, it will be turned in at the end of the experiment to be graded. A new notebook for each experiment, or perhaps better a single bound notebook with pre-numbered pages, should be used. The notebook should contain detailed descriptions of what you are doing, and should also pose questions to be answered, make predictions, and contain a dialog with yourself. In other words, make the notebook an integral part of the experimentation process. As you work and make notebook entries, record the date and time. Write clearly, and make drawings whenever possible. Use pen and do not erase mistakes, but line them out neatly.

A forty-point notebook rubric might look like...

Objective Basic (3) Cool (4) Far Out (5)
Descriptions of Work at least a few whole sentences careful but sparse fun and informative
Data: Testable by Others some data convincing data lots of convincing data
Question papertrail: Active dropped question, here and there stream of consciousness waterfall of inquisitivity
Predictions/Models: Clear/Useful occasional calculation drawings and derivations too "aha!" experiences for the reader
Dates: Admissible in Court course & semester noted dated entries in sequence pages initialed & sealed
Timing: Real or After Fact? unlikely written in one day realtime flavor, including deadends clearly an unfolding account
Index: Clues for a Re-Visit page numbers are used findable highlight pagelist content list at start or finish
Source Citations: Connected credit given where due findable citations included bibliography too

Notes about the patent perspective on documenting scientific discovery can be found here.


Sample Short Talk Rubric

For our advanced physics lab course, these should be about six minutes long and include the following:

a). Tell the audience what you are going to say, say it, then summarize at the end.

b). Use an overhead projector with transparencies, or presentation software, as these are the most generally available ways of presenting material. Transparencies can be made with copier, computer printer, or simply by hand.

c). Choose topics that will be informative and interesting, and related to the experiment at hand. However don't choose so wide a subject that you cannot keep the talk within the 6 minute target. Stick with that chosen topic.

d). Following each talk, be prepared to answer questions and participate in a discussion about what you have presented. Those in the audience will be expected to do the same, and possibly to help in providing some constructive evaluation of the presentations as well.

A talk rubric for peer evaluation, using a forty-point scale, might look like...

Objective Basic (3) Cool (4) Far Out (5)
Structure: Outline/Delivery/ReCap two of three parts all three parts all three, in harmony
Data: Relevant/Reproducible some data convincing data many facts to report
Interpretation/Theory: Clear/Testable some explanation clear plausible explanation of data data convincingly explained
Discussion/Conclusions: Useful/Correct something to conclude something useful to conclude useful and original results
Delivery: Captivating/Apropo listening did not cause sleep that was interesting that was fun
Timing: On Schedule/Not Rushed almost done on time done on time nicely done on time
Visual Aids: Effective/Pleasing adequate helpful memorable
Audience Rapport: Responsive/Astute positive eye contact good eye contact and listening dynamic interaction

Sample Scientific Report Rubric

When used for our advanced physics laboratory course, these are due 1 week after the experiment is completed. Is this also a reasonable rubric for possible submissions to the LosAlamos/Cornell eprint arXiv? How might we make it even more appropriate for that purpose? Getting back to guidelines for our advanced lab:

a). Be terse and only a few pages in length.

b). Include enough information that the experiment can be repeated by an independent student observer who is at a junior physics major level.

c). Use a simple format - Abstract, Introduction, Experimental Details, Your Observations, Interpretation/Theory and Discussion/Summary.

d). The American Institute of Physics "Style Manual" and/or a sample from Physical Review can be used as a guide.

e). Include in the paper some reference to contemporary research (less than 10 years old) related to the experiment, if you can locate such references.

A forty-point report rubric might look like...

Objective Basic (3) Cool (4) Far Out (5)
Format: Abstract, Intro, Setup, Observe, Interpret, Conclude four of six clearly separate five separate functioning sections all six sections in harmony
Abstract: Specific Results short interesting abstract fact-rich in addition also has original science
Experiment and/or Observations: Testable describes methods, tools, & your observations also fact-rich, interpretation-free adequate for others to check results
Interpretation/Theory: Useful section separate from observations clear models and believable interpretation original in context of the literature
Discussion/Conclusions clear summary & conclusions brief, with speculations labeled as such complementary to other sections
Delivery: Clear to Many readable whole sentences short sentences with minimal jargon understandable by anyone
Figures: Helpful at least one graphic a figure comparing data to theory memorable graphical insights
Literature Refs: Connected credit given where due APS format bibliography all citations findable a year from now

Sample nanoWorld webQuest Rubric

This is a scientific diary and report of one of your adventures as a nano-detective. We recommend that it have six sections, in the following order:

1). Work Log: This is a simple diary of project activity with entries of the form "date-time/what happened".

2). Introduction: This section describes what you can find out about the challenge before actually doing the experiment. Here references to external sources are crucial.

3). Method: Describe here how you took data, with enough detail that another student in the course could repeat your experiment and get approximately the same results.

4). Data/Observations: This section reports the actual measurements that you made. These might be typed directly into the text, or simply summarized in the text and provided separately in spreadsheet or image attachments.

5). Interpretation/Theory: In this section, you make the connection between your observations and the objective of the nanoworld challenge. What conclusions are possible, based on your observations?

6). Assumptions/Uncertainties: Here is where you address the strength of your interpretation. What assumptions are involved, and how likely are they to be valid? If you have quantitative conclusions, provide evidence concerning the expected numerical error in those conclusions. This might also be a good place to mention possible future work.

A generic nanoWorld webQuest rubric, using a forty-point scale, might look like...

Objective Basic (3) Cool (4) Far Out (5)
Work Log: Storyline at least a few whole sentences readable but terse fun and informative
Work Log: Detail one or two dates timeline is apparent clearly unfolding account
Intro: Informative conventional wisdom a reference or two literature well-considered
Method: With Style some clues provided tells others how clear and original
Data: Testable by Others a few observations some reproducible data lots of convincing facts
Interpretation: Clear/Useful connects data to project goals relevance is crystal clear "aha!" experiences for the reader
Uncertainties: Realistic a guess at the errors reality check for the reader well established report limits
Delivery: Clear to Many readable whole sentences short sentences with minimal jargon understandable by anyone

Here are: a web page with some links to nanoWorld webQuests, a draft nanoexercise for introchem, the summary of a proposal about their possible applications, and a review form for fleas on fleas.


Sample Peer Review Rubric

Note: This rubric evaluates the reviewer, not the manuscript submitted. In that context, the most important part of a peer review are the reviewer's suggestions for ways to improve in the "essay portion" of the review. In general, the more (i) homework you do in the review process, (ii) time spent learning about the subject/literature referenced, and (iii) suggestions offered the better even if your feedback at first glance seems annoying to the author. Your suggestions might best be conveyed as items referenced to specific sentences, pages, and/or parts of the presentation. In the case of submitted manuscripts, they also might be conveyed as "markup" or marginal notes.

A final step in a peer review is "filling out the form". This usually includes making recommendations to the editor, as well as some sort of ranking of various aspects of the paper (as for instance occurs in the presentation rubrics above). A sample form for the non-essay component of a peer review, used by a scientific journal, may be found here. This step addresses issues of importance to the editor, but is generally of limited use to the author.

A generic peer-review rubric, which uses a forty-point scale and assumes that the reviewed-report itself is evaluated according to an 8-element rubric like those above, might look like...

Objective Basic (3) Cool (4) Far Out (5)
Report-Rubric Eval: Completeness six of eight items addressed seven of eight all eight items addressed
Report-Rubric Eval: Justification some mention in the essay rubric observations justified full and constructive treatment
Context: Why Publish? mention of work's context value and quality addressed clearly clear awareness of relevant history
Language: Clarified? managed to improve something readability clearly improved constructive input every other sentence
Missing: Omissions Reported? addressed what was missing clearly identified omissions guidance for filling in the blanks
Trim: Fat Found? mentions if anything is too much made a case for what stays and goes clearly and convincingly
Contribution: Useful? at least read the paper clearly did some homework earned an acknowledgement if not co-authorship
Delivery: Clear? readable whole sentences short sentences with minimal jargon understandable by anyone

Sample Test Essay Rubric

Here the goal is to provide structured feedback to students, e.g. on their input to a 5-point essay question, with a string of 4 numbers whose average is the point-score for the problem. We do it here in terms of the "four C's": Completeness, Correctness, Clarity, and Creativity. Thus a grade written as: 4352 → 3.5 means that, for this person's answer, Completeness was 4 (most underlying issues addressed), Correctness was 3 (answers weren't too far off), Clarity was 5 (quite unambiguous), but Creativity was 2 (same old same old). This is a compact way to convey perceived strengths and weaknesses, even if the grading process for such questions is inherently subjective.

A generic test essay rubric, for use with a 5-point question as described above, might look like...

Objective Basic (3) Cool (4) Far Out (5)
Completeness some of the underlying issues were addressed most of the underlying issues were addressed all of the underlying issues were addressed
Correctness the answers, in so far as they could be understood, were not too far off some comments are dead on everything said was on the money
Clarity the ambiguity is tolerable some of the things said are clear wrong or right, at least we know what you were trying to say
Creativity haven't heard it said exactly that way before at least one new perspective offered the whole thing was a breath of fresh air

Weekly Discussion Board Activity

Note: This rubric is designed to evaluate participation e.g. in weekly topic discussion fora as part of "one room schoolhouse" courses like the web/lab course at UM-StL on "emergent nanoscience practicals". Like the essay rubric, a subset of the issues evaluated are subjective but can nonetheless be characterized. In addition, however, web discussion packages generally assist with some numerical measures of participation as well. Some of these are grouped at the beginning in the table below.

A generic forum-participation rubric, which uses a forty-point scale and assumes that the reviewed-report itself is evaluated according to an 8-element rubric like those above, might look like...

Objective Basic (3) Cool (4) Far Out (5)
Number of Posts three or more six or more nine or more
Number of New Threads one or more two or more three or more
Length of Reply Threads one or more, on average two or more, on average three or more, on average
Reader Evaluation Stars two stars, on average three stars, on average four or more stars, on average
Topic Coverage of Posts some of the issues were addressed many topics were touched on no stone left uncovered
Correctness of Posts the answers, in so far as they could be understood, were not too far off some comments are dead on everything said was on the money
Clarity of Posts the ambiguity is tolerable some of the things said are clear wrong or right, at least we know what you were trying to say
Creativity of Posts haven't heard it said exactly that way before at least one new perspective offered the whole thing was a breath of fresh air

One-line Quantitative Puzzler

Here, we're working on a rubric to evaluate one-line quantitative problems proposed for use in a course with few math pre-requisites. We'd like these one-liners to be (a) relevant to and do-able with the course material that it targets, (b) useful and/or interesting to those taking the class, (c) understandable, and (d) original. Suggestions for this rubric are invited. As with the test essay rubric above, taking the average of numbers for the four items below puts the evaluation on a 5-point scale. For example, a point assignment of "4353" for the items below gives the problem a score of (4+3+5+3)/4 = 3.75 out of 5.

Quantitative one-liners for various sections of our how things work class might, for example, include "What voltage between you and a doorknob is implied by a [d] cm spark?", "How deep is a hole that takes a coin dropped on earth [t] seconds to hit bottom?", "How many 'gees' of centripetal acceleration does it take to keep you moving in a circular path of radius [r] meters at a speed of [v] meters per second?", and "By what factor does the heat that an object radiates change when that object's absolute temperature changes by a factor of [f]?".

Objective Basic (3) Cool (4) Far Out (5)
Relevance has something to do with the target content possible to do if you know the target content important to be able to do for the course
Useful or Interesting the question won't put everyone to sleep hey, knowing how to answer this might come in handy wow, I can't wait to tell all my friends
Clarity someone might be able to make sense of the problem parts of this question should be easy for all to understand excellent work letting folks know what solving this problem involves
Originality haven't heard it said exactly that way before at least one new perspective offered the whole thing was a breath of fresh air

This page (http://newton.umsl.edu/~philf/rubrics.html") is Copyight (1999-2005) P. Fraundorf, and hosted by UM-StL Physics and Astronomy. Average whole-site page requests est. around 2000/day hence more than 500,000/year. Requests for a stat-counter linked subset of pages since 4/7/2005: .