Best Practices for Remote Examinations

Published 04/02/20. Last updated 10/16/20.

Delivering remote examinations presents a number of challenges, from technical execution to academic integrity. The following presentation covers three key aspects of remote examinations:

Preparing students for remote exams

Safeguarding academic integrity

Deploying different types of exams

The presentation draws on the Guidance and Recommendations for Instructors and Students on Proctoring and Final Examinationsby the Working Group on Online Examinations and Proctoring along with Recommendations from the Task Force on Instructional Resilience. It also includes text from Tips for Assessing Student Learning Remotely by the UC Berkeley Department of Economics, prepared by Frederico Finan in June 2020.

I. PREPARING STUDENTS FOR REMOTE EXAMS

Remote exams pose a number of challenges. Students may not understand the technology; downloading and uploading materials can be a problem, especially for students with poor internet connections; and there is a risk that the accessibility of students with disabilities will be compromised. In addition, assessment practices vary widely across courses and instructors. Consequently, it is critical that instructors take the time to explain how remote exams will work, set student expectations, and test the assessment technologies that they intend to deploy.

1. Assess student needs and DSP accommodations prior to the exam

We recommend that instructors review their students’ DSP accommodations, potential internet connectivity issues, workspace conditions, and geographic location, then make a plan to address their students’ needs. Instructors should consult with DSP Disability Specialist Services in a timely manner if they have concerns about providing an accommodation. DSP guidelines for remote instruction and assessment can be found here. Members of the Working Group on Online Examinations and Proctoring have developed a sample google survey that can be modified to meet an instructor’s needs.

Some DSP students may not be able to access the exam platform with their assistive technology. In such a situation, instructors need to provide an alternate way for DSP students to submit their exams.

2. Provide students with an opportunity to test drive the online submission process

Instructors should test the online submission process of their exams at least one week before the exam date. This will help troubleshoot any potential issues and allow the instructor to receive student feedback. Instructors should keep the dry run as close to the actual exam format as possible.

Both bCourses and Gradescope allow an instructor to duplicate assignments. This function can be used to create the dry run assignment with a similar setup.

In addition, in both bCourses and Gradescope, instructors can make duplicate courses and import their teaching staff as students. By viewing the course from the perspective of the students, instructors and teaching staff can determine whether there are any problems with the process of taking and submitting exams.

Duplicating a course in bCourses
Make a new course in Gradescope
bCourses also supports viewing your course as a Test Student

If any DSP students indicate that they cannot upload an exam by following the available instructions, an alternate way for them to submit the exam must be provided.

3. Use flexible exam timing when appropriate and provide additional time for downloading exams and uploading solutions

It is possible that students will be located across several time zones. Instructors are encouraged to offer either multiple exam times or a time-limited exam that students can choose when to take over a longer period (e.g., a 1 hour exam that can be taken during a 4 hour window). It is important to realize that exams and answers take time to both download and upload, especially when using an online platform such as Gradescope or bCourses. Instructors are encouraged to consider downloading and uploading time when determining both the length and time window of the exam.

Alternatively, some instructors decide to offer two exam times approximately 12 hours apart, so that students in diverse time zones will all have a viable exam option. In this case, students should be asked ahead of time which time slot they will use to take their exam. One of these exam times should be the original exam time.

4. Provide clear written guidance to students when collaboration is permitted

Expectations for what constitutes "allowable collaboration" should be clearly outlined in advance and repeated at the beginning of the assignment or exam. If the exam, project, or assignment includes sections for which collaboration is allowed along with sections that should be completed individually, these sections should be clearly delineated. When allowing collaboration, students should be required to name their collaborators and detail the specific contributions of each to the final product.

5. Provide clear written guidance on how to contact instructors or GSIs during an exam

A GSI or instructor should be available for the entire exam time, including extended time for students with DSP accommodations. Communicate to students as early as possible what the protocol will be for asking for help or raising concerns to an instructor or GSI. Course staff should have a plan for when and how to refer any issues to the instructor.

Where possible, make use of communications tools with which students are already familiar, unless new tools provide necessary features. We also recommend using only bCourses or third-party tools (such as Gradescope) that are supported by the campus. TopHat and Piazza are not supported by the campus and, therefore, should not be used.

Students should have an opportunity to ask questions privately (e.g. via Zoom breakout rooms or private chat), while instructors should have a method by which corrections and clarifications can be made available to the entire class (e.g. a screen- shared announcements document).

II. SAFEGUARDING ACADEMIC INTEGRITY

The move to online assessments has multiplied the opportunities for academic dishonesty. Students are often not proctored directly, making it is difficult to monitor use of notes or technology. Many students feel pressure to cheat on the assumption that if they do not, they will be disadvantaged relative to students who do. Although there is no perfect solution to academic dishonesty, actions can be taken to mitigate the risk and protect those students who do behave with integrity. This section presents some general approaches, while Section III discusses specific exam formats that can reduce the opportunities and incentives for academic dishonesty.

1. Communicate the Honor Code

It is important to remind students of their obligation to act with integrity. While moral suasion may not be enough by itself to prevent cheating, it can help foster a healthier ethical climate. The UC Berkeley Honor Code is short, clear, and aspirational.

“As a member of the UC Berkeley community, I act with honesty, integrity, and respect for others.”

We recommend including the Honor Code on the cover sheet or at the beginning of exams. This practice should be combined with discussions or communications to ensure that students in the class understand what constitutes a violation of this code when taking an exam. The tone of the conversation should clearly convey the expectation that students adhere to the Honor Code.

Beneath the Honor Code sentence, instructors may wish to add their own expectations for the students tailored to the framework of the exam, for example:

I alone am taking this exam.
I will not receive assistance from anyone while taking the exam nor will I provide assistance to anyone while the exam is still in progress.
Other than with the instructor and GSI, I will not have any verbal, written, or electronic communication with anyone else while I am taking the exam or while others are taking the exam.
I will not have any other browsers open while taking the exam.
Unless I have been authorized to use assistive technology, I will not use any external devices, such as phones, tablets, or additional computers, during the entire period this test is available for students to take.
I will not refer to any books, notes, or online sources of information while taking the exam, other than what the instructor has allowed.
I will not take screenshots, photos, or otherwise make copies of exam questions to share with others.

Instructors may also consider spelling out the penalties for academic dishonesty, for example:

I understand that violating the honor code on this exam will result in a grade of “F” for the exam.
I understand that violating the honor code on this exam will result in a grade of “F” for the course.
I understand that violating the honor code on this exam will result in my referral to the Student Conduct Office for further disciplinary action, possibly including suspension or expulsion.

Instructors may choose to have students sign a copy of the Honor Code at the beginning and/or end of the exam.

2. Design assessments to reduce the incentives for misconduct and perceptions of misconduct

The most effective way to safeguard academic integrity is to deploy forms of assessment that are less susceptible to cheating than written exams: oral exams, papers, group projects, poster sessions, discussion boards, etc. If written exams must be administered, there are ways to discourage, if not eliminate, cheating and to mitigate its effects on other students. This section presents some general principles; Part III discusses a variety of specific exam formats and how they reduce the incentives and opportunities for cheating.

Do not curve exams. The perception that others in a class are benefiting from misconduct can undermine the culture of integrity that allows a fair exam process, creating an incentive for misconductIt is important for students to perceive that they will not be disadvantaged by the misconduct of other students, a problem that can be exacerbated by grading on a strict curve. In the absence of a curve, students who cheat may still get A’s, but their “success” will not reduce the chances for other students to get A’s as well.
Make exams open-book, so that students who consult notes and books do not gain an unfair advantage over students who adhere to closed-book rules.
Schedule multiple, short, low-stakes tests, rather than one or two lengthy, high-stakes exams. Students may be less tempted to cheat if the stakes of the exam are relatively low and less able to cheat if the exams are of short duration. In addition, this approach is generally recognized as superior for promoting learning and retention.
Formulate questions that require more than simple memorization of basic facts. Multiple-choice, true-or-false, and fill-in-the blanks questions that have a single, short, correct answer are the most susceptible to cheating, as students can quickly look up answers online and/or share answers with other students in the class. Such questions also test a relatively limited understanding of course content. By contrast, more complex questions that require a deeper understanding and mastery of course concepts do not lend themselves as easily to cheating since there is no single, simple, correct answer. Examples of such questions include requiring students to resolve complex problems, respond to fictional scenarios, or present and defend a subjective position. This approach is not only better at preventing cheating, but also promotes a higher form of learning than rote memorization.
Use randomized oral exams to ensure that the work that students have submitted is truly their own. Prior to the examination, a random set of integers can be generated that are then matched to an alphabetized list of students from the course. After the exam and prior to assigning final grades, all of the previously selected students may be questioned about their solutions to the exam. For this process to be utilized, it should:

Be announced to students sufficiently far in advance of the examination for students to self-identify to DSP and seek accommodations from that office.
Make clear the consequences for a student who fails to appear for the oral examination.
Exhibit no bias and should be administered in such a manner as to be defendable to other instructors.

For additional information on the use of randomized oral exams, see item #7 here

3. Be mindful of online resources that enable academic misconduct

Instructors should be mindful of sites such as Chegg and discussions on Reddit that can facilitate academic misconduct. If instructors reuse problems from textbooks or previous tests on final examinations, it is likely that students will be able to quickly find the solutions to these questions online. Some sites, including Chegg, have policies against posting answers to exam questions (Chegg honor code) and will take down offending material if asked. However, in almost all cases, by the time this happens, the answers will have been circulated widely and the academic integrity of the exam compromised.

4. Report student misconduct to the Campus’s Center for Student Conduct

Following the Berkeley Campus Code of Student Conduct, it is vital for instructors to report cases of misconduct to the Center for Student Conduct (CSC). A report should be filed even if the student admits responsibility and accepts the sanctions imposed by the instructor. To detect repeated violations, the CSC relies on reporting from instructors.

CSC data is reviewed by the administration and senate and helps to identify systemic misconduct. First-time offenses are often treated as an educational intervention, but repeated academic conduct violations are treated seriously. It is also possible that the case is due to a misunderstanding: the CSC provides a process through which such cases can be resolved in a just manner. CSC procedures are described on the CSC website

5. Consider whether Zoom proctoring is right for your class

Under certain circumstances, exams in summer 2020 and fall 2020 may be proctored via Zoom. For details, see the EVCP’s Remote Proctoring Policy for Summer Sessions and Fall Semester. Instructors planning to use Zoom proctoring are urged to read the policy carefully and make careful note of their responsibilities under the policy, most notably regarding student privacy and the accommodation of students with disabilities. In addition, instructors should understand that no technology is a panacea and that Zoom alone will not put an end to academic dishonesty. It remains essential for instructors to deploy a variety of tools to foster a climate of integrity and reduce the rewards and opportunities for cheating.

III. DEPLOYING DIFFERENT TYPES OF EXAMS

Instructors should clearly explain the format of exams to students well in advance. One aim is to reduce the anxiety many students are feeling about the uncertainty of how remote exams are going to work. Keep in mind that remote exams are not only unfamiliar but also likely inconsistent across different classes.

Some of our recommendations involve using Question Banks. Instructions on constructing these resources can be found via canvas and Haas

The different exam formats discussed below are designed to be robust vis-à-vis technological glitches, to minimize opportunities for academic misconduct, and, in many cases, to assess students’ deeper understanding of the course content, as opposed to rote memorization.

1. Open book/open notes exams

Some considerations in designing these exams include:

Time limits: the more limited the time, the less a student can rely on searching through materials and the more they will need to study in advance in order to complete the exam.
Course material: to deter students from turning to Google or outside sources or assistance, write questions that are closely and specifically connected to course content (e.g., compare the evidence used by Author A and Author Z; apply concept B to the data presented in article Y). This can also be done in auto-gradable questions (multiple choice. T/F, etc.) by constructing scenarios to which students must apply knowledge derived from course lectures and readings.
Embedded Conceptual Assessment: in seeking to assess students’ ability to use certain concepts or analytical skills from the class, an instructor could provide a previously unseen document, source, dataset, etc., and ask how it could be used to support or contradict arguments made by Scholar C and Scholar X.
Applied Questions: writing questions centered on fictional scenarios or actors provides a mechanism to assess how deeply students understand specific ideas while limiting the ability of students to turn to the internet for answers. Examples might include wholly or partially invented chemical compounds, math formulae, plants, locations, or characters.
Randomized Exams: The order of exam questions can be randomized, more than one version of an exam can be used with different questions or question organization, or exams can use various combinations of both.
The platform bCourses allows instructors to build multiple Question Banks Multiple ‘Question Banks’ can be utilized for the same exam to divide sections of the test. Both the answers and question order can be randomized in bCourses. A large Question Bank allows for each student to receive a randomized subset of questions from the bank, so that every student is receiving a different test.

2. Timed examinations

Both bCourses and Gradescope support setting a time limit for an assignment. Both platforms make it possible to input extensions for students who receive accommodations. Students in other time zones can be individually assigned to a more favorable time slot in the details section of Quiz in bCourses. For instructions, see bCourses timed quizzes and Gradescope timed assignments

When using timed examinations, give a “starting window” of at least 15 minutes so that students do not need to rush to take advantage of the full time allotted to them. For large courses, a 15-minute starting window will also alleviate potential sources of instability on web services.

3. Exams in which the order of questions and answers are shuffled

The platform bCourses allows instructors to easily randomize all question types as well as the answer order for multiple-choice questions. To randomize answers, select ‘Shuffle Answers’ in the Quiz details (below). Note: instructors cannot use answer choices such as ‘D) both A and C’ because randomizing answer choices means that A and C will not be the same for all students.

Screenshot - Shuffle Answers

4. Exams in which students receive different versions of the exam (via question banks)

Creating different but equitable versions of an exam - and telling students in advance - is a method of deterrence by essentially informing students that they are better off focusing on their own exam and not wasting precious time trying to contact fellow students to compare questions and answers. Depending on the nature of the exam, instructors do not necessarily have to create entirely different versions. Indeed, instructors might be able to produce this effect, with less effort on the part of the instructional staff, by having different versions of just a small part of the exam.

One way that instructors can construct different versions of the exam using bCourses is to populate questions in a Question Bank and have students answer a subset of these questions. An exam that includes a Question Bank can randomly select a subset of the questions for each student (for example, a random subset of 25 of the 100 total questions). Initially, the instructor can set the individual point value for all questions in the Question Bank to 1 point, then can adjust the point value later when building the exam and adding a ‘+New Question Group’.

Individual exams can use more than one Question Bank. For example, an exam could have three sections drawn from independent Question Banks: Honor Code; multiple choice; essay (see below). To construct such an exam, first, store the questions in the Question Banks. When building the exam in bCourses ‘Quiz’, select your questions from ‘New Question Group’.

Screenshot - Question Bank

5. Exams in which written performance can be confirmed with oral questioning

An instructor may want to use oral questioning to confirm written exam performance when the instructor has reason to believe that a student has cheated. Instructors are authorized by the campus’s Student Code of Conduct to conduct such an inquiry regarding academic integrity. For instance, instructors could orally confirm the demonstrated written performance of all students whose exam solutions suggest cheating, if this is done in a way that is based solely on student answers and is "blind" to the identity of the student. Different students could be asked about different problems on the exam, so long as the process exhibits no impermissible bias. Students could also be questioned by requesting a written response, instead of an oral response. If the subsequent inquiry causes the instructor to believe that cheating occurred, then the allegation of academic misconduct would be resolved in accordance with the requirements of the campus Student Code of Conduct

Instructors who are considering this sort of systematic review or exam performance are advised to seek guidance from their Chair and/or the Office of Legal Affairs regarding their specific post-exam protocol.

6. Exams for students with Disabled Students' Program (DSP) accommodations

Instructors can either (1) enter alternate durations in the details section of a bCourses Quiz using the “Moderate Quiz” function to assign specific students more time, or (2) duplicate a prepared exam in Quiz and make adjustments for DSP accommodations. Duplicate exams can be set up with longer exam duration (150%, 200% time, etc.) then assigned to individuals with specified levels of accommodation. Grades from the DSP-specific exam will need to be imported once more into Gradebook for the entire class examination to correctly calculate course grades for the accommodated students.

7. Final papers/take-home essay exams

Many of the suggestions for open-book exams apply to final papers and take-home exams, understood to be papers/exams that students have more than 24 hours to complete. Additional considerations and suggestions include:

Required use of (only) course materials: If the final paper/take-home exam is based entirely on course materials or course materials and additional information/sources provided by the instructor, it can useful to insist that students only use materials from the class and/or certain course materials and/or a minimum number of course materials (possibly from different segments of the class). This serves both to assess their learning of course material and to deter using assignments from the internet.
Cover sheets: A cover sheet provided by the instructor that lists the requirements/acceptable materials and requires students both to indicate that they have complied and to document what they used can be useful. It can deter the use of prohibited external sources, create a check for students to ensure that they have met the requirements, and reduce time spent by readers/GSIs/instructors checking exam compliance.
Assignment types: It can be helpful to identify course learning goals in both content and skills, and mix and match a certain skill with specific content to ensure the exam/paper assesses what the student should have learned this semester. In addition to more typical explanatory or argument-driven essay exams, curation assignments, applied assignments, and mock peer review assignments can offer opportunities to assess student learning.
Curation assignments ask students to collect and select a limited number of examples tied to a theme (the more specific, the better - or an instructor can set a broad topic that students can narrow according to their interests). The basis could be course materials or broader. It can be helpful to identify an imagined audience beyond the instructor (campus, 5th graders, state legislators, museum goers, an interest group, etc.). The assignment could then require a companion essay that uses the course materials to frame, explain, justify, and analyze the selected options. A corollary is a timeline assignment in which students have to pick X number of most important events/figures/issues connected to a course theme and explain/justify the selection.
Applied assignments may take the form of a policy memo, a letter to a corporate board, a briefing for a legislator, etc. The task ultimately requires students to synthesize course material to make an argument about what could or should be done, drawing on research results, conceptual tools, analytic skills, and course materials.
Mock peer review assignments give students an article/chapter/proposal related to the course and assign them the task of assessing its strengths and weaknesses and making recommendations using the ideas and concepts they learned in the class. This could be an article or chapter that students have not seen before, a mock research paper or grant proposal written at a level that makes sense for the class, or an assessment of whether a source fits an imagined anthology or whether a piece of scholarship should be taught in a future iteration of the class.
Questions centered on fictional (or unique) scenarios or actors provide a mechanism to assess how deeply students understand specific ideas while limiting the ability of students to turn to exam services or the internet for answers. Examples might include wholly or partially invented chemical compounds, math formulae, plants, locations, or characters.

8. Final reflections

In lieu or as part of a standard final exam, instructors might consider requiring a short paper reflecting on what the student learned over the course of the semester. Emphasizing specificity and the use of examples can create quite meaningful documentation of learning.

Possible questions include:

What have you learned this semester?
What terms and concepts do you now recognize or understand differently?
What terms, concepts, and ideas are more complicated now?
What are you now able to explain (to your family, to a friend, to a roommate, to anyone else you may encounter)?
How do you read/think about current events differently?
How can you use information and ideas from this class now and in the future?

Possible requirements include:

Reference a minimum of [X] course materials in the reflection.
Select a recent news article/op-ed/tweet and use it to structure the student’s reflection. Are there terms the student reads differently? Can the student situate the issues raised in different or more complex ways? If the student were the editor, how would they push the writer to clarify or extend the article? If the student met the writer, how would this class inform questions the student might ask?
The student could reread 1-3 course readings and discuss how they read them differently now compared to the beginning of this class. What terms or references can the student identify or situate? What connections can the student make to what they have learned?
The student could review an assignment from earlier in the course and revise it. What can they now do better, more efficiently, more clearly, etc. than at the beginning of this class? Why? What helped them improve over the semester?
Identify a challenge the student faced in this course, explain the challenge, identify what they did to overcome it, and reflect on what it means about this course’s material/design/goals. What can the student carry with them into the future from the class?

9. Presentations

Faculty may administer a graded student presentation, including as an alternative assignment to a final exam when appropriate, so long as students are not required to use video or penalized for choosing not to do so, and the format is not in conflict with any DSP accommodations. Screen sharing of materials (e.g., PowerPoint slides) via Zoom during the presentation is allowed.

Compiled by Jonah Levy and members of the Undergraduate Council of the Berkeley Division of the Academic Senate and members of the Joint Senate-Administration Instructional Resilience Task Force. Helpful comments from Frederico Finan's "Tips for Assessing Student Learning Remotely" were also incorporated.