Python Autograder documentation

class code_feedback.Feedback()

Class to provide user feedback and correctness checking of various datatypes, including NumPy arrays, Matplotlib plots, and Pandas DataFrames.

classmethod add_feedback(text)

Adds some text to the feedback output for the current test.

classmethod call_user(f)

Attempts to call a student defined function, with any arbitrary arguments specified in *args and **kwargs. If the student code raises an exception, this will be caught and user feedback will be given.

If the function call succeeds, the user return value will be returned from this function.

classmethod check_dataframe(name, ref, data, subset_columns=[], check_values=True, allow_order_variance=True, display_input=False)

check_dataframe Checks and adds feedback regarding the correctness of a pandas DataFrame. Author: Wade Fagen-Ulmschneider (waf)

By default, checks if the student DataFrame data contains the same contents as the reference DataFrame ref by using pandas.testing.assert_frame_equal after basic sanity checks.

Parameters:

  • name, String: The human-readable name of the DataFrame being checked
  • ref, DataFrame: The reference (correct) DataFrame
  • data, DataFrame: The student DataFrame
  • subset_columns = [], Array of Strings: If subset_columns is an empty array, all columns are used in the check. Otherwise, only columns named in subset_columns are used in the check and other columns are dropped.
  • check_values = True, Boolean: Check the values of each cell, in addition to the dimensions of the DataFrame
  • allow_order_variance = True, Boolean: Allow rows to appear in any order (so long as the dimensions and values are correct)
  • display_input = False, Boolean: Display the student’s answer in the feedback area.

classmethod check_list(name, ref, data)

Check that a student list has correct length with respect to a reference list. Can also check for a homogeneous data type for the list.

  • name: Name of the list that is being checked. This will be used to give feedback.
  • ref: Reference list.
  • data: Student list to be checked. Do not mix this up with the previous list! This argument is subject to more strict type checking.
  • entry_type: If not None, requires that each element in the student solution be of this type.
  • accuracy_critical: If true, grading will halt on failure.
  • report_failure: If true, feedback will be given on failure.

classmethod check_numpy_array_allclose(name, ref, data, accuracy_critical=False, rtol=1e-05, atol=1e-08, report_success=True, report_failure=True)

Feedback.check_numpy_allclose(name, ref, data)

Check that a student NumPy array has similar values to a reference NumPy array. Note that this checks value according to the numpy.allclose function, which goes by the following check: absolute(a - b) <= (atol + rtol * absolute(b))

  • name: Name of the array that is being checked. This will be used to give feedback.
  • ref: Reference NumPy array.
  • data: Student NumPy array to be checked. Do not mix this up with the previous array! This argument is subject to more strict type checking.
  • rtol: Maximum relative tolerance between values.
  • atol: Maximum absolute tolerance between values.
  • accuracy_critical: If true, grading will halt on failure.
  • report_failure: If true, feedback will be given on failure.

classmethod check_numpy_array_features(name, ref, data)

Check that a student NumPy array has the same shape and datatype as a reference solution NumPy array.

  • name: Name of the array that is being checked. This will be used to give feedback.
  • ref: Reference NumPy array.
  • data: Student NumPy array to be checked. Do not mix this up with the previous array! This argument is subject to more strict type checking.
  • accuracy_critical: If true, grading will halt on failure.
  • report_failure: If true, feedback will be given on failure.

classmethod check_numpy_array_sanity(name, num_axes, data)

Perform a sanity check on a NumPy array, making sure that it is in fact defined and has the correct dimensionality. If the checks fail then grading will automatically stop.

  • name: Name of the array that is being checked. This will be used to give feedback.
  • num_axes: Number of axes that the array should have.
  • data: NumPy array to check.

classmethod check_plot(name, ref, plot, check_axes_scale)

Checks that a student plot has the same lines as a reference plot solution. Can optionally check the axis scales to ensure they are the same as the reference.

  • name: Name of plot scalar that is being checked. This will be used to give feedback.
  • ref: Reference plot.
  • data: Student plot to be checked. Do not mix this up with the previous value! This argument is subject to more strict type checking.
  • check_axes_scale: One of None, ‘x’, ‘y’, or ‘xy’. Signals which axis scale should be checked against the reference solution.
  • accuracy_critical: If true, grading will halt on failure.
  • report_failure: If true, feedback will be given on failure.
  • report_success: If true, feedback will be given on success.

classmethod check_scalar(name, ref, data)

Check that a student scalar has correct value with respect to a reference scalar. This will mark a value as correct if it passes any of the following checks:

  • abs(ref - data) < ref(ref) * rtol
  • abs(ref - data) < atol

One of rtol or atol can be omitted (set to None) if that check is unwanted. Or both, but then nothing would be graded :)

  • name: Name of the scalar that is being checked. This will be used to give feedback.
  • ref: Reference scalar.
  • data: Student scalar to be checked. Do not mix this up with the previous value! This argument is subject to more strict type checking.
  • accuracy_critical: If true, grading will halt on failure.
  • rtol: Maximum relative tolerance.
  • atol: Maximum absolute tolerance.
  • report_failure: If true, feedback will be given on failure.
  • report_success: If true, feedback will be given on success.

classmethod check_tuple(name, ref, data)

Check that a student tuple has correct length with respect to a reference tuple, and same values.

  • name: Name of the tuple that is being checked. This will be used to give feedback.
  • ref: Reference tuple.
  • data: Student tuple to be checked. Do not mix this up with the previous tuple! This argument is subject to more strict type checking.
  • accuracy_critical: If true, grading will halt on failure.
  • report_failure: If true, feedback will be given on failure.
  • report_success: If true, feedback will be given on success.

classmethod finish(fb_text)

Complete grading immediately, additionally outputting the message in fb_text.

static not_allowed()

library_function = Feedback.not_allowed

Used to hook into disallowed functions, raises an exception if the student tries to call it.

classmethod set_score(percentage)

Set the score for the test case, should be a floating point value between 0 and 1.


class pl_unit_test.PLTestCase(methodName='runTest')

Base class for test suites, using the Python unittest library. Handles automatic setup and teardown of testing logic.

Methods here do not need to be overridden by test suites.

classmethod get_total_points()

Get the total number of points awarded by this test suite, including cases where the test suite is run multiple times.

classmethod setUpClass()

On start, run the user code and generate answer tuples.

classmethod tearDownClass()

Close all plots and increment the iteration number on test finish


class pl_unit_test.PLTestCaseWithPlot(methodName='runTest')

Test suite that includes plot grading. Will automatically check plots for appropriate labels.