Feat/api adrs course videos - #39102
Draft
Abdul-Muqadim-Arbisoft wants to merge 3 commits into
Draft
Abdul-Muqadim-Arbisoft wants to merge 3 commits into
Abdul-Muqadim-Arbisoft wants to merge 3 commits into
Conversation
Adds a conforming REST surface for a course's video assets at
/api/authoring/v1/courses/{course_key}/videos/ and .../{edx_video_id}/,
as one ViewSet with list, retrieve, create and destroy.
The legacy GET at /api/contentstore/v0/videos/uploads/{course_id}/{edx_video_id}
declares a video id and ignores it, returning the whole course listing. The new
version splits that in two: the collection endpoint succeeds the behaviour the
legacy route really has, and the member endpoint finally honours the id.
Three defects are fixed on the new addresses only, because fixing them in place
would break anyone parsing the current responses:
- The legacy GET and HEAD write. They flip every video stuck in 'upload' for
more than 24 hours to 'upload_failed'. The new read path computes the same
display status without the write; the legacy route still reconciles.
- Three incompatible error shapes, including a 400 with an empty body whose
payload rides in the HTTP reason phrase, become one error envelope.
- A non-JSON Accept header returns a redirect to the authoring MFE, or a 500
when COURSE_AUTHORING_MICROFRONTEND_URL is unset, which is the platform
default. The new addresses are JSON only.
Business logic is reused by importing the existing video_storage_handlers and
edxval.api functions; none of it is duplicated or edited. Authentication drops
to the platform default, which refuses inactive users on the new addresses
while the legacy addresses are unchanged.
Addresses under a course that no endpoint serves are answered with the error
envelope rather than the site's HTML error page, so a client can read them.
Part of openedx#39060.
The CMS schema trimmed '/api/contentstore' from every path and advertised a
matching server entry. Once the document also carries paths under another
prefix, that arrangement has no server that serves both: a client generated
from the advertised server builds
https://{CMS_BASE}/api/contentstore/api/authoring/v1/courses/{key}/videos/
and gets a 404.
Trimming is therefore off and the server list is reduced to the public host and
the Studio host, so one server URL serves every operation in the document. Path
strings change from /v0/... to /api/contentstore/v0/..., but no effective URL
moves and no operation is added or removed by this commit.
Two consequences worth knowing. The schema comparison gate reads a path-key
rename as a removal, so it reports a breaking change for each renamed path; the
compensating addition is invisible to it. And AUTHORING_API_URL now joins full
paths rather than shortened ones, so it must be the Studio service root rather
than a gateway base.
These settings predate this work; the defect is latent on master and is
surfaced, not introduced, by adding a second prefix to the document.
The three video upload operations under /api/contentstore/v0/videos/uploads/ now have conforming successors, so they are marked deprecated in the published schema by a post-processing hook, and their docstrings name the successor address for each operation. The hook lives outside the frozen version directory, and the change inside it is docstring text only: no import, no statement, no decorator, so the v0 endpoints behave exactly as before. No runtime Deprecation header is emitted, since that would be a response change on a live endpoint. The pre-processing filter is widened at the same time so the authoring paths reach the document at all; without it they are dropped silently. Still owed before the deprecation window can formally open: a DEPR issue in openedx/public-engineering, and a named removal release. Both need someone who knows the current release train. Part of openedx#39060.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standardizes the Course Videos API (#39060, umbrella #38137) onto a conforming, versioned surface, and leaves the existing
/api/contentstore/v0/videos/uploads/…endpoints working exactly as they are today.New addresses:
GET | POST /api/authoring/v1/courses/{course_key}/videos/GET | DELETE /api/authoring/v1/courses/{course_key}/videos/{edx_video_id}/One
viewsets.ViewSetwithlist,retrieve,create,destroy, backed by a service layer that composes the existingvideo_storage_handlersandedxval.apifunctions by import. No business logic is duplicated, and none of it is edited.Why a new version rather than in place. Three of the four contracts had to change to conform, and each change is breaking for anyone parsing the current responses: the list gains a pagination envelope, every non-2xx gains a single error envelope in place of three incompatible shapes, and
POSTanswers 201 instead of 200. ADR 0037 puts all three on a new version. The v0 surface is frozen and keeps its exact bytes.Why the GET becomes two endpoints.
GET /api/contentstore/v0/videos/uploads/{course_id}/{edx_video_id}declares a video id and ignores it, returning the whole course listing. The new version splits that: the collection endpoint succeeds the behaviour the legacy route really has, and the member endpoint finally honours the id.Change table
POST …/videos/files[]entries, typed request serializerGET …/videos/{"videos": […]}, unpaginated?ordering,?view=minimalGET …/videos/{edx_video_id}/DELETE …/videos/{edx_video_id}/Three defects fixed, on the new addresses only
uploadfor more than 24 hours toupload_failed(Video.save()from a GET). The new read path computes the same display status without the write. The legacy route still reconciles, and a test pins that it does.Acceptheader returns a redirect to the authoring MFE, or aTypeError500 whenCOURSE_AUTHORING_MICROFRONTEND_URLis unset — which is the platform default.ADRs applied
help_texton every field,content_typechoices derived from the handler's real setIsAuthenticated+ the existingHasCourseAuthorAccess; every legacy check keeps a home@extend_schemaper action,ErrorResponseSerializeron every documented 4xx, both response variants publishedRetrieveAPIView+DestroyAPIViewpair and aCreateAPIView; one recorded deviation (routing form, below)POST /generate_video_upload_link/{course_key}is a genuine action-URL sibling hitting the same handler, but it is outside this issue's endpoint table and needs its own DEPR — recorded as a follow-upDefaultPaginationvia the library'sIterablePaginationMixin; not retrofitted to v0, where it would be breakingorderingis serializer-validated againstedxval's own sort fieldsauthentication_classesdeclaration anywhere; Bearer gone by constructionCOURSE_AUTHORING_MICROFRONTEND_URLis a redirect to a screen, not a configuration payload, and is not carried forward?view=minimalwith both representations published;?fields=deliberately omitted, one mechanism being enough here/api/authoring/v1/courses/{course_key}/videos/[{edx_video_id}/], the ADR's owncontentstore→authoringtargetScopedQuerysetMixinwraps a view-levelget_queryset(), and this ViewSet has none — the only realQuerySetis built insideedxval. Hand-writing a pass-through policy would be worse than not having one. The three visibility layers are enumerated belowstatus, no locale-dependent enum valueCompliance matrix
help_texton all fields;allow_nullon the fields null in practice;content_typechoices fromVIDEO_SUPPORTED_FILE_FORMATSCourseVideoSchemaTest::test_the_declared_full_representation_is_the_one_returned;test_create_rejects_unsupported_content_typeIsAuthenticated+ reusedHasCourseAuthorAccess; Studio read check and the upload-pipeline gate preserved in the service and translated at the boundaryCourseVideoAuthorizationTest— 8 allow rows, 32 deny rows@extend_schema; every documented 4xx$refsErrorResponse; 404examplescarry both causes; enum post-processing preservedCourseVideoSchemaTest(12 tests),CmsSchemaHookTest,CmsSchemaAddressTesthandle_videos's method/Acceptdispatch split. Deviation:path()+.as_view({…})rather than a router registration — ADR 0038's own conforming-URLconf example (docs/decisions/0038-…rst:245-259) uses exactly this form for a course-nested resource keyed by an opaque idCourseVideoQueryCountTest(literal counts);CourseVideoUrlContractTestStandardizedErrorMixinfirst in the MRO; unresolvable addresses under a course answered as JSON;ParseError/MethodNotAllowed/NotAcceptable/UnsupportedMediaTypecataloged; Django-nativePermissionDenied/Http404translated at the service boundary;errorsvalues are message stringsCourseVideoRequestErrorTest;CourseVideoUrlContractTest::assert_json_not_found;test_create_names_the_entry_each_message_belongs_tosend_video_status_update; legacy keeps the writeCourseVideoReadOnlyTest(incl. HEAD on both addresses and the legacy-route companion)DefaultPaginationviaIterablePaginationMixin; defaults, clamping and bounds pinnedtest_list_returns_ten_videos_a_page_by_default;test_list_accepts_the_maximum_page_size;test_list_reduces_an_oversized_page_size_to_the_maximum;test_the_declared_list_envelope_is_the_one_returnedorderingallow-list generated fromedxval.api.VideoSortField, default-created, total order via edxval'sedx_video_idtie-break so pagination is stabletest_list_honours_ordering;test_list_rejects_unknown_ordering;test_list_defaults_to_newest_firstauthentication_classesline, including on the fallback viewtest_the_viewset_does_not_declare_authentication_classes(checks__dict__, so inheritance cannot satisfy it)test_the_legacy_accept_header_branch_is_not_carried_over?view=minimalviaMinimalViewMixin; both variants published as aoneOf; default shape is the legacy rowtest_both_video_representations_are_declared;test_the_declared_minimal_representation_is_the_one_returnedv1/for this resource; v0 frozen; deprecation markers are docstring text only. Missing: DEPR issue link, named removal releasecourse_keyconverter, opaqueedx_video_id, snake_case version-free namesCourseVideoUrlContractTestget_queryset()to wrap. Layer 1 endpoint access =permission_classes; layer 2 record visibility = the service's pipeline gate + edxval'scourse_idandis_hidden=Falsefilters; layer 3 =ordering/page/page_size/view, sort-and-shape only, provably unable to widen the row settest_list_excludes_other_courses;test_list_excludes_removed_videos;test_retrieve_404_for_video_of_another_coursestatus;status_nontranslatednot carried forwardtest_the_status_is_never_locale_translatedBackward compatibility
v0/videos/*routes, the three/api/contentstore/v1/videos/…routes, and all seven unversioned Studio routes.contentstore/views/tests/test_videos.py+rest_api/v1/views/tests/test_videos.py).reverse()caller and no path literal in-repo;frontend-app-authoring@mastercalls the legacy Studio/videos/{courseId}route and/api/contentstore/v1/videos/…, nevervideos/uploads.openedx/openedx-platform-sdkdoes not exist.Intentional differences for a caller migrating v0 → v1
Each is asserted by the parity harness, which fails if a declared difference does not occur:
{"videos": […]}statusuntranslated;status_nontranslatedabsentcreatedkeeps microsecond precision (legacy's encoder truncated to milliseconds)Accept-driven representation switchingOrg/Course/Runkeys 404 at routingSessionAuthenticationAllowInactiveUser. A deactivated global-staff account gets 200 on v0 today and 401 on v1. Pinned in both directions bytest_inactive_global_staff_is_refused/test_inactive_global_staff_still_reaches_the_legacy_addressIn-place edits
rest_api/v0/views/authoring_videos.pygit diff --numstat= 28 insertions, 0 deletions; AST proof: deleting every docstring node from base and head yields byte-identical ASTscms/urls.pyregister_url_converters()and theapi/authoring/v1/mountcourse_keyandusage_key; neither collides with any existing converter, and a full warning capture across setup and resolver walk produced no re-registration warningcms/lib/spectacular.pycms/envs/production.py,cms/envs/devstack.pyPOSTPROCESSING_HOOKS(re-including the default enum hook); trimming turned off;SERVERSreducedCmsSchemaAddressTest; see belowTwo new modules beside the ViewSet, both inside the pre-existing
v1/:error_types.py(registers the four uncataloged DRF request errors) andviews/unknown_route.py(answers an unmatched address under a course with the standard envelope instead of Studio's HTML 404 page).Error-format decision
Versioned, not in place. Clients parsing v0's three shapes are unaffected because v0 is untouched, so no in-place client evidence is owed.
Deprecations
deprecated: trueon the three v0 upload operations, set by the post-processing hook incms/lib/spectacular.py(outside the frozen directory), plus docstring markers naming each successor. No runtimeDeprecationheader: that is a response change on a live frozen endpoint.Not yet supplied, and owed before merge:
Both need someone who knows the current release train. The docstrings say "will be removed in a future release", matching the merged v0-xblock precedent (#38723), and that precedent carries no issue link or release either — but the ADR text is stricter than the precedent, which is why the 0037 box above is unticked.
The unversioned Studio routes (
/videos/{course_key},/generate_video_upload_link/{course_key}) are not deprecated here: they are what the MFE actually calls, and two of their operations have no conforming successor yet.Authorization
IsAuthenticated(from@view_auth_classes())permission_classeshas_course_author_access→ 403HasCourseAuthorAccess, view level, all four actionshas_studio_read_access→PermissionDenied_get_and_validate_course; now translated to a DRF exception at the service boundary so it yields a realauthzenvelopevideos/uploads-not-configuredcourses__course_id,is_hidden=False)Deny-path tests, each × all four operations: anonymous → 401; authenticated with no role → 403;
CourseLimitedStaffRole→ 403; author of another course → 403;ccx-v1:key → 403; plus "no domain record is touched when refused". Every 4xx asserted withassert_error_envelope.Gate report
No gate was skipped.
Why
schemafails. This PR stops the CMS schema trimming its path prefix, so every published path string changes from/v0/…to/api/contentstore/v0/…. The gate classifies a path-key rename as a removal and cannot see the compensating addition, so it reports 56 BREAKING. Verified mechanically on the gate's own artifacts: 56 removed, 0 of them without a full-path counterpart, and exactly 2 genuinely new paths (the two authoring addresses). No operation was deleted and no effective URL moved — 81 operations before, 85 after, key sets otherwise identical.The trim was removed because admitting
/api/authoring/into the document while trimming only/api/contentstoreproduced a document where the new operations were unreachable from the advertised server: a generated client builthttps://{CMS_BASE}/api/contentstore/api/authoring/v1/courses/{key}/videos/. TheSCHEMA_PATH_PREFIX_TRIM/SERVERSsettings are not new — they have been on master since #33694 (2023) — so this is a latent defect this PR surfaces and fixes rather than one it introduces.schema_diff.py check-presentnow passes and the mixed-prefix warning is gone.Sanctioned hygiene WARN.
CourseVideoPagination(DefaultPagination)is a local subclass of a library primitive.edx-drf-extensions10.8.0 contains noget_paginated_response_schema, soDefaultPaginationdescribes only 4 of its own 7 envelope keys; the subclass adds that one method so the published schema matches the body actually returned.Follow-ups
openedx/public-engineeringfor the three v0 methods, and the named removal release (see Deprecations).edx-drf-extensions: addget_paginated_response_schematoDefaultPagination— every Open edX list endpoint currently publishes a 4-key envelope schema for a 7-key body. This PR's subclass deletes itself when that lands.edx-drf-extensions: addParseError,MethodNotAllowed,NotAcceptable,UnsupportedMediaTypeto the builtin error catalog, and makeflatten_detailstop stringifying the detail dict intodetail. Registered locally here; the durable fix is upstream.AUTHORING_API_URLchanges meaning: it was joined to shortened paths and is now joined to full paths, so it must be the Studio service root rather than a gateway base.cms/envs/mock.ymlcarries the gateway-shaped example. Any deployment setting it to a gateway base should re-point it.lms/envs/common.pystill trims/api/enrollment). No mix today because its filter admits only that prefix; the next prefix added there hits it.POST /generate_video_upload_link/{course_key}is a genuine ADR 0031 / 0038 r10 candidate hitting the same handler; outside this issue's table, needs its own DEPR.courses/{course_key}/videos/for the video resource.tags=["openedx-platform-sdk"]./api/authoring/v1/no/such/thing/) still gets Studio's HTML 404. ADR 0038 r8 rejects any route whose first segment after the version is a placeholder, so a conforming route cannot claim it; this matches every other API mount in the platform.Sequencing
PR #39078 will also create
v1/authoring_urls.py, theapi/authoring/v1/mount,register_url_converters()incms/urls.pyand the widened filter. It has not landed, so this PR creates them in the same shape. On rebase: take #39078's version of each shared file and re-apply the two route additions.app_name = "authoring_v1"is assumed to match its namespace.