You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CodeTailor: accept ===== before test code, and support subgoal labels - #1599
1. Accept four or more = before a question's test code.
For book-authored questions, CodeTailor found the unit tests by splitting the activecode source on exactly ====. activecode.js accepts ===== and longer, which older books use, and for those questions the split left a stray = at the top of the tests. Every generated solution then failed with a SyntaxError, and CodeTailor silently fell back to the static example instead of personalizing help. CodeTailor now splits on a line of four or more =, matching activecode.js. This commit stands alone and can be merged separately.
2. Subgoal labels.
Instructors can write subgoal labels (full-line comments naming each step) into a question's backup Parsons problem. Students can write their own in the editor.
Help before any code: a student who asks for help before writing any code gets a Parsons problem of just the backup's labels, under the function's def line from the starter code. The def is #settled in the settled-block personalization. Untouched starter code counts as no code. This is logged as solution type subgoal_labels.
Student labels: when a student's code contains labels, the LLM is asked to follow that plan and keep them. They're restored into the personalized solution and puzzle, each above the code it names. Labels are also copied onto paired distractors, so they never reveal which block is correct.
What isn't a label: commented-out code (# total = 0) isn't treated as a label, comments that ship with the starter code aren't the student's labels, and full-line comments never become draggable blocks.
Unchanged for existing questions: questions whose backup problem has no comments behave exactly as before.
Testing:
Tested by hand on a local server with py4e-int's list_writeAvgq:
the labels-only puzzle (graded correct),
personalized puzzles that keep a student's labels,
AI_personalized results once the separator issue was out of the way.
CodeTailor found a book-authored question's unit tests by splitting its
activecode source on exactly "====". activecode.js accepts "=====" and
longer, which older books use, and for those questions the split left a
stray "=" at the top of the tests. Every generated solution then failed
with a SyntaxError and CodeTailor silently fell back to the static example
instead of personalizing help.
Split on a line of four or more "=", as activecode.js does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Instructors can now write subgoal labels -- full-line comments naming each
step -- into a question's backup Parsons problem, and students can write
their own in the editor.
- A student who asks for help before writing any code gets a puzzle of just
the backup problem's labels, under the function's def line from the
starter code (locked in place for the settled-block personalization).
Untouched starter code counts as no code, so the print call a question
ships with no longer sends the student to the LLM. Logged as solution
type "subgoal_labels".
- When a student's code contains labels, the LLM is asked to follow that
plan and keep them, and they are restored into the personalized solution
and puzzle, each above the code it names. Labels are copied onto paired
distractors too, so they never reveal which block is correct.
- Comments never become blocks. Commented-out code ("# total = 0") is not
a label, and comments that ship with the starter code ("# write code
here") are not the student's labels.
- The personalized solution no longer has blank lines where comments were
stripped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups:
- Find the test suffix the way activecode.js does: the first run of four
or more "=", wherever it is, not only on a line of its own.
- Keep a "===!" visible suffix in the starter code. activecode drops only
the marker, so cutting the starter there made an untouched exercise look
edited and skip the labels-only puzzle.
- Look up the starter code in the course's base course first, then fall
back to a global name match, as the main CodeTailor path does. Question
names are unique only within a base course.
- Treat any commented-out Python statement other than a bare expression
as code, e.g. "# count: int".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Java branch treats common commented-out code without a trailing semicolon/brace as a label. For example, // if (x > 0), // int total = 0, and // return total all make is_label_line(..., "java") return true, so backup code can appear as a subgoal and student code can be preserved as a supposed plan. Expand the Java code heuristic to recognize control/declaration/assignment/return syntax, not only line terminators.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits.
1. Accept four or more
=before a question's test code.For book-authored questions, CodeTailor found the unit tests by splitting the activecode source on exactly
====. activecode.js accepts=====and longer, which older books use, and for those questions the split left a stray=at the top of the tests. Every generated solution then failed with aSyntaxError, and CodeTailor silently fell back to the static example instead of personalizing help. CodeTailor now splits on a line of four or more=, matching activecode.js. This commit stands alone and can be merged separately.2. Subgoal labels.
Instructors can write subgoal labels (full-line comments naming each step) into a question's backup Parsons problem. Students can write their own in the editor.
defline from the starter code. Thedefis#settledin the settled-block personalization. Untouched starter code counts as no code. This is logged as solution typesubgoal_labels.# total = 0) isn't treated as a label, comments that ship with the starter code aren't the student's labels, and full-line comments never become draggable blocks.Testing:
Tested by hand on a local server with py4e-int's
list_writeAvgq:AI_personalizedresults once the separator issue was out of the way.🤖 Generated with Claude Code