Skip to content

fix: keep SSO library classes and generic signatures in release builds - #8

Merged
pbek merged 1 commit into
qownnotes:mainfrom
MySkeletonHurts:fix/sso-generic-signature-r8
Sep 17, 2026
Merged

pbek merged 1 commit into
qownnotes:mainfrom
MySkeletonHurts:fix/sso-generic-signature-r8

Conversation

@MySkeletonHurts

Copy link
Copy Markdown
Contributor

Addresses the CapabilitiesApi.getCapabilities() / "empty capabilities response" failure
described in #6 (that issue also covers an unrelated SQLiteBlobTooBigException, which looks to
already be fixed separately — not touched here).

Summary

Adding a Nextcloud account fails in release builds only, with "Nextcloud returned an empty
capabilities response" — reproducible every time from a clean install. The identical debug build
works fine against the same server and account, including full note sync afterward.

Root cause

NextcloudRetrofitServiceMethod (in the Nextcloud SSO library) decides how to deserialize a
response by reflecting on the API interface method's generic return type
(method.getGenericReturnType()), expecting Observable<ParsedResponse<OcsResponse>>. Our
proguard rules kept our own org.qownnotes.mobile.backend.nextcloud.** classes, but declared
nothing for the SSO library's own classes (com.nextcloud.android.sso.**, which ships no consumer
proguard rules) and no -keepattributes Signature. R8 was therefore free to rename/optimize the
SSO library's classes, which corrupted the nested generic signature: ParsedResponse<OcsResponse>
eroded down to a raw ParsedResponse, losing <OcsResponse>.

When that erosion happens, NextcloudRetrofitServiceMethod's instanceof ParameterizedType check
on the inner type fails, and it falls through to a documented fallback path that Gson-parses the
response as ParsedResponse itself (not OcsResponse) and unwraps .getResponse() early — which
comes back empty, since the real JSON body has no response field at that level. That's what
surfaces as "Nextcloud returned an empty capabilities response".

Confirmed directly with a temporary diagnostic log reading
CapabilitiesApi::class.java.getMethod("getCapabilities").genericReturnType from inside our own
code:

  • Before: genericReturnType=a6.b<e5.k> ... innerArg=class e5.k innerIsParameterizedType=false
  • After: genericReturnType=x5.b<com.nextcloud.android.sso.api.ParsedResponse<org.qownnotes.mobile.backend.nextcloud.OcsResponse>> ... innerIsParameterizedType=true

And via the R8 mapping file: the failing runtime path was landing in
NextcloudRetrofitServiceMethod.lambda$invoke$2/$4 (the fallback's .map(r -> r.getResponse())),
not the direct-cast path used when the type is correctly recognized.

Fix

-keepattributes Signature

-keep class com.nextcloud.android.sso.** { *; }
-keep interface com.nextcloud.android.sso.** { *; }

Both lines were verified individually necessary through a fast build → sign → install →
diagnostic-log loop. Other candidate changes tried along the way (dropping allowoptimization on
the existing **Api interface rule, adding *Annotation*,InnerClasses,EnclosingMethod to
-keepattributes) produced byte-identical R8 output and were reverted — this is the minimal diff.

Test plan

  • Reproduced the failure on the official signed v0.4.0 and v0.5.0 releases, and on a
    locally-built release variant (via just build-fdroid, then self-signed with a throwaway
    local keystore so it could be installed) without this fix, from a completely clean install.
  • Applied the fix, rebuilt the same release variant, self-signed the same way, installed fresh
    (both QOwnNotes variants uninstalled first).
  • Confirmed account setup succeeds and notes sync fully, matching debug-build behavior.
  • ./gradlew spotlessCheck test lintDebug :app:licensee all pass unchanged.

@pbek

pbek commented Sep 17, 2026

Copy link
Copy Markdown
Member

Sorry, this PR seems to have slipped past me. 🤔

@pbek pbek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The release-only SSO failure is credible, and the linked device-test failure is unrelated: that job stopped in android-actions/setup-android@v3 because sdkmanager tools no longer exists, before any emulator or instrumentation test ran. Please rebase onto current main, which already uses setup-android@v4 with packages: platform-tools, so CI can exercise the tests.

I do have one change request for app/proguard-rules.pro:11-12: please narrow the blanket com.nextcloud.android.sso.** keep rule to the reflection-sensitive types/members proven necessary. Keeping the entire namespace disables shrinking, optimization, and obfuscation for all SSO implementation code. In an apples-to-apples release build at this PR's base, it increased the unsigned APK from 5,447,603 to 5,929,123 bytes, about 482 KB / 8.8%. The mapping confirms that the relevant behavioral change is keeping NextcloudRetrofitServiceMethod and ParsedResponse, while the current rule also retains otherwise removable SSO classes and members.

Please also correct the comment at lines 7-10. Android-SingleSignOn 1.3.4 does ship consumer rules (proguard.txt from consumer-proguard-rules.pro), and those rules already contribute -keepattributes Signature; this is visible in the baseline merged R8 configuration. The missing piece is retaining the relevant generic-signature endpoint under R8 full mode, rather than the library shipping no consumer rules.

Local verification: the release build and spotlessCheck test lintDebug :app:licensee pass, and the PR merges cleanly with current main.

Adding a Nextcloud account (and every subsequent sync) failed in release
builds only, with "Nextcloud returned an empty capabilities response".
Debug builds worked fine against the same server and account.

The Nextcloud SSO library's NextcloudRetrofitServiceMethod inspects the
generic return type of API interface methods reflectively
(method.getGenericReturnType()) to decide how to deserialize responses,
expecting Observable<ParsedResponse<OcsResponse>>. In release builds this
resolved to a raw ParsedResponse with the <OcsResponse> argument erased,
so the reflective check silently took a fallback path that deserializes
the response as ParsedResponse itself instead of OcsResponse, producing
an empty result.

Two things were needed to fix it:
- -keepattributes Signature, so R8 keeps the generic signature metadata
  reflection depends on at all.
- Keep rules for NextcloudRetrofitServiceMethod and ParsedResponse
  specifically. Android-SingleSignOn 1.3.4 ships its own consumer
  proguard rules (which already cover -keepattributes Signature), but
  they don't keep these two classes, so R8 full mode is still free to
  rename/optimize them, which corrupts the nested generic signature when
  remapping it.

Verified with a locally-signed release build: without this change the
account add reproduces the failure every time; with it, account setup
and sync work identically to a debug build.
@MySkeletonHurts
MySkeletonHurts force-pushed the fix/sso-generic-signature-r8 branch from 53b2f8d to 7f8c57c Compare September 17, 2026 17:37
@MySkeletonHurts

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review! I pushed an update addressing all three points:

  • Rebased onto current main.
  • Narrowed the proguard rule to just com.nextcloud.android.sso.api.NextcloudRetrofitServiceMethod and ParsedResponse instead of the whole com.nextcloud.android.sso.** namespace. Unsigned release APK went from 5,929,123 bytes back down to 5,682,095 bytes.
  • Corrected the comment. I did leave the actual declaration -keepattributes Signature just defensively in case the library's consumer rules ever drop it, but let me know if you want me to remove the declaration too.

Re-verified locally: spotlessCheck, test, lintDebug, :app:licensee all pass, and a signed release build with the narrower rule still fixes the original account-add failure on-device.

@pbek

pbek commented Sep 17, 2026

Copy link
Copy Markdown
Member

Thank you!

@pbek
pbek merged commit e4c81e8 into qownnotes:main Sep 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants