Frequently Asked Questions

Only the student or CALI will be able to see the score, with two exceptions:


  • The first exception is when the student explicitly indicates that they want to submit the score for review by a faculty member. This would be in conjunction with the LessonLink and/or AutoPublish services. You can learn more about these in their respective FAQs.
  • The second exception is that CALI will use the scores as a way to improve its lessons without revealing any personal detail about the student. When used in aggregate, scores can be used by CALI to determine if there are improvements to be made to a CALI lessons. The aggregate scores of all students who take a lesson may be shared with the lesson's author to see if there are ways to refine or improve the lesson for future students.

CALI Lessons are best suited to interactive questions that engage and challenge students to learn and to think about the material.

One technique to account for different levels of student learning, different learning styles, and to provide more teaching opportunities, is to create branching questions. Branching questions also allow use of interactive questions to replicate the Socratic questioning style used in the law school classroom. Branching questions can appear at any point in a Lesson and make it more individualized for students.  For example, branching questions can be used to address specific strengths and weaknesses that each student brings to the Lesson. Any one Lesson may contain one or more question models.

CALI has identified five models of question formats: (1) Nintendo-ing, (2) Expert mode; (3) Novice Mode; (4) Stick to your response; and (5) Fill out your knowledge/Remedial branch. All of these formats are useful in interactive materials.

 

A Lesson utilizing questions in a basic building block format is the most common, and the most linear, style of question. Here, regardless of the student's answer, the student progresses to the next question. CALI has dubbed this style of questioning "Nintendo-ing," since the student simply clicks through the Lesson in a linear manner. This question style is very useful for review material, or places where we want to drill students on their understanding of concepts underlying any issue.

 

 

In the "Expert" mode questions are still presented in a linear fashion. However, when students answer a particular question correctly, they skip ahead in the Lesson. Thus, students who correctly answer a "threshold" question about the material are not bored by a review of the initial concepts or basic materials. Instead, they can move directly to the more challenging questions. Consequently, in "Expert" mode students' progress can be individualized to match their skills and understanding of the material. Moreover, based on the method of naming pages, students do not necessarily know that they have skipped ahead, or conversely been taken the slow route through the Lesson. This style also allows for the student to have a different experience with the Lesson the second or third time he works through the material.

As you can see, this style of questioning can be used to replicate the style of teaching you might use in the classroom. In a typical classroom you might ask students a question to gage their general understanding of the material. Based on the class' response you will either move forward with the material (the "Expert" mode) or re-examine some of the more basic material.

 

 

The "Novice" mode is related in theory and structure to the "Expert" mode. Here too students move through the Lesson in an individualized fashion. In contrast to the "Expert" mode though, students who answer a threshold question incorrectly are directed to a remedial branch where the exact root of their misunderstanding or their deficiencies can be explored. For example, suppose students have a common misconception about joint tenancy. A question in the Lesson can ask them about this topic. Students answering with the common misconception can be directed to a tutorial designed to correct their mistaken understanding. Students who understand the misconception and answer the question correctly progress through the Lesson without having to do any of the extra questions. And, as with any other question model, based on the method of naming pages, students may not know that they have been diverted to refresher or more basic material.

 

"Stick to your response" emulates a technique faculty often use in the classroom of introducing possibly conflicting material, or additional facts, into students' analysis. In this question model a student who selects the correct response to the question will branch to an additional line of questioning that can inquire whether the student took factor 'X' into consideration when selecting his answer. As a result, a student's response can be challenged and analyzed. Here too, faculty can address misconceptions, gray areas of the law, or weaknesses in students' analysis. Finally, like any of the other question models, "Stick to your response" can be combined with other question formats. For example, students who select the correct response would branch to a series of questions that are directed to a higher level of analysis and that explore a more difficult concept. In contrast, students who select the wrong response would branch to a series of "Remedial" questions designed to reinforce the underlying theories and basic concepts of the problem.

 

The final question model combines features of the previous four models and adds something new. Like the "Novice" mode, students who answer incorrectly branch to a series of questions designed to work on their fundamental knowledge of the material. And while students giving the correct answer branch to a series of questions like the "stick to your response" model, here students are not pressed to defend their response. Instead, the student's knowledge and understanding is expanded by questions that delve deeper into the material.

 

No, at this time CALI Lesson scores cannot be deleted by students or CALI staff.

CALI Lesson scores remain available for the life of a CALI account.


Scores from the current semester can be viewed on a student's "My Lesson Runs" page. Previous semester lesson scores will be found in chronological order on this page under the "Past Lesson Runs" tab.

CALI Lesson scores are saved automatically, even if you don't remember to click the "Exit & Resume Later" link at the top of the page within each CALI Lesson." However, they will not be reported unless you click the "finalize my score" button.

Generally speaking, faculty are unable to view student performance on CALI Lessons. There are three exceptions:

  1. Students provide the faculty member with a printed or digital version of the certificate that appears at the end of Lessons. This will provide only the score for the Lesson.
  2. The faculty member creates a LessonLink and can view the scores and question by question performance directly via the "Current LessonLinks" tab for LessonLinks created that semester, provided that they are logged in with the faculty account that created the LessonLink.
  3. The faculty member creates a QuizWright quiz or uses CALI Author to create a self-published lesson and can view student scores and question by question performance via the "My CALI Library" link from the CALI Dashboard menu.

For information please visit the FAQs on LessonLink, QuizWright, and self-published lessons.

 

 

Strictly speaking, you can't change the score of a lesson once you have answered all the questions. You can change answers, but the original score will remain.

If you have not finalized a lesson and still have questions to answer, you can resume and finish the CALI Lesson. This will change the final score of the lesson. You are also able to take CALI Lessons as many times as you wish.

Please note: if you are taking a LessonLink CALI Lesson or an AutoPublished Lesson, your professor will be able to see multiple lesson runs. For more information on resuming a lesson, LessonLink or AutoPublish, please see their respective questions in the FAQs.

Generally speaking, only the student knows which CALI Lessons a student has taken and what their score was. However, if a faculty member would like to use CALI Lessons for formative or summative assessment, they have a few options to find student scores.


  1. They can have students submit a lesson certificate that shows their completion and performance on a CALI Lesson. Each certificate lists the students' name, when they finalized the lesson, how many questions were attempted and how many they answered correctly.
  2. If you use LessonLink, you will be able to view the scores and question-by-question performance of your students on a CALI Lesson.