diff --git a/.nx/version-plans/add-experimental-android-native-coverage.md b/.nx/version-plans/add-experimental-android-native-coverage.md new file mode 100644 index 00000000..c68923d3 --- /dev/null +++ b/.nx/version-plans/add-experimental-android-native-coverage.md @@ -0,0 +1,5 @@ +--- +__default__: minor +--- + +Harness now offers experimental native Android coverage for selected Gradle modules, so you can see which native Kotlin/Java code paths your Harness tests exercise. After a covered run, Harness produces `native-coverage.lcov`, giving you a concrete way to inspect and report native coverage alongside your existing test results. diff --git a/packages/config/src/types.ts b/packages/config/src/types.ts index fc0d5143..e2409c08 100644 --- a/packages/config/src/types.ts +++ b/packages/config/src/types.ts @@ -118,6 +118,18 @@ export const ConfigSchema = z ), }) .optional(), + android: z + .object({ + modules: z + .array(z.string()) + .min(1, 'At least one Gradle module path is required') + .describe( + 'Gradle module paths to instrument for native code coverage, ' + + 'e.g. [":android"]. The app must be built with the harness coverage ' + + 'init script to enable JaCoCo offline instrumentation.' + ), + }) + .optional(), }) .optional() .describe('Native code coverage configuration.'), diff --git a/packages/coverage-android/README.md b/packages/coverage-android/README.md new file mode 100644 index 00000000..50ded6bf --- /dev/null +++ b/packages/coverage-android/README.md @@ -0,0 +1,107 @@ +![harness-banner](https://react-native-harness.dev/harness-banner.jpg) + +### Experimental Android Native Coverage for React Native Harness + +[![mit licence][license-badge]][license] +[![npm downloads][npm-downloads-badge]][npm-downloads] +[![Chat][chat-badge]][chat] +[![PRs Welcome][prs-welcome-badge]][prs-welcome] + +⚠️ **EXPERIMENTAL** ⚠️ + +`@react-native-harness/coverage-android` adds native Android code coverage collection for React Native Harness. It uses JaCoCo offline instrumentation to instrument selected Gradle modules, collects `.ec` execution data files from the app during test runs, and writes a `native-coverage.lcov` report after the run finishes. + +Coverage collection is supported on **Android emulators and physical devices** (debug builds only). + +## Installation + +```bash +npm install --save-dev @react-native-harness/coverage-android +# or +pnpm add -D @react-native-harness/coverage-android +# or +yarn add -D @react-native-harness/coverage-android +``` + +After installation, rebuild the app with the coverage init script (see Usage). + +## Usage + +Build the app with JaCoCo offline instrumentation: + +```bash +cd android +./gradlew assembleDebug \ + --init-script ../node_modules/@react-native-harness/coverage-android/scripts/harness-coverage-init.gradle \ + -PHarnessCoverageModules=:mylib +cd .. +``` + +Add the modules you want to instrument in `rn-harness.config.mjs`: + +```javascript +import { androidPlatform, androidEmulator } from '@react-native-harness/platform-android'; + +export default { + runners: [ + androidPlatform({ + name: 'android', + device: androidEmulator('Pixel_8_API_35'), + bundleId: 'com.example.app', + }), + ], + coverage: { + native: { + android: { + modules: [':mylib'], + }, + }, + }, +}; +``` + +Run Harness with coverage enabled: + +```bash +react-native-harness --coverage --harnessRunner android +``` + +When coverage is collected successfully, Harness writes `native-coverage.lcov` to the project root. + +## How it works + +- A Gradle init script applies JaCoCo offline instrumentation to compiled Kotlin/Java class files +- Injects a ContentProvider that bootstraps a coverage flush helper on app startup +- The helper writes JaCoCo execution data (`.ec` files) to app internal storage every second +- After tests, Harness pulls `.ec` files from the device, merges them, and generates LCOV + +## Requirements + +- Android SDK with emulator or physical device +- Java 11+ (for JaCoCo CLI) +- Android runner configured with `@react-native-harness/platform-android` +- Debug build of the app using the coverage init script +- `@react-native-harness/coverage-android` installed (provides the init script and runtime helpers) + +## Limitations + +- Experimental and subject to change +- Requires building with the Gradle init script (`--init-script`) +- Coverage collection writes reports to the project root +- Build and test environments must share access to the build output (original class files + `jacococli.jar`) + +## Made with ❤️ at Callstack + +`@react-native-harness/coverage-android` is an open source project and will always remain free to use. If you think it's cool, please star it 🌟. [Callstack][callstack-readme-with-love] is a group of React and React Native geeks, contact us at [hello@callstack.com](mailto:hello@callstack.com) if you need any help with these or just want to say hi! + +Like the project? ⚛️ [Join the team](https://callstack.com/careers/?utm_campaign=Senior_RN&utm_source=github&utm_medium=readme) who does amazing stuff for clients and drives React Native Open Source! 🔥 + +[callstack-readme-with-love]: https://callstack.com/?utm_source=github.com&utm_medium=referral&utm_campaign=react-native-harness&utm_term=readme-with-love +[license-badge]: https://img.shields.io/npm/l/@react-native-harness/coverage-android?style=for-the-badge +[license]: https://github.com/callstackincubator/react-native-harness/blob/main/LICENSE +[npm-downloads-badge]: https://img.shields.io/npm/dm/@react-native-harness/coverage-android?style=for-the-badge +[npm-downloads]: https://www.npmjs.com/package/@react-native-harness/coverage-android +[prs-welcome-badge]: https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=for-the-badge +[prs-welcome]: ../../CONTRIBUTING.md +[chat-badge]: https://img.shields.io/discord/426714625279524876.svg?style=for-the-badge +[chat]: https://discord.gg/xgGt7KAjxv diff --git a/packages/coverage-android/android/src/main/AndroidManifest.xml b/packages/coverage-android/android/src/main/AndroidManifest.xml new file mode 100644 index 00000000..6ad3cb4d --- /dev/null +++ b/packages/coverage-android/android/src/main/AndroidManifest.xml @@ -0,0 +1,9 @@ + + + + + diff --git a/packages/coverage-android/android/src/main/kotlin/com/harness/coverage/CoverageHelper.kt b/packages/coverage-android/android/src/main/kotlin/com/harness/coverage/CoverageHelper.kt new file mode 100644 index 00000000..11e4d877 --- /dev/null +++ b/packages/coverage-android/android/src/main/kotlin/com/harness/coverage/CoverageHelper.kt @@ -0,0 +1,62 @@ +package com.harness.coverage + +import android.app.Activity +import android.app.Application +import android.content.Context +import android.os.Bundle +import android.util.Log +import java.io.File + +object CoverageHelper { + private const val TAG = "HarnessCoverage" + private var ecFile: File? = null + private var timer: java.util.Timer? = null + private var cachedAgent: Any? = null + + fun setup(context: Context) { + val agent = try { + Class.forName("org.jacoco.agent.rt.RT") + .getMethod("getAgent") + .invoke(null) + } catch (e: Exception) { + Log.w(TAG, "JaCoCo agent not available — was the app built with coverage?", e) + return + } + cachedAgent = agent + + val pid = android.os.Process.myPid() + ecFile = File(context.filesDir, "coverage-$pid.ec") + + val app = context.applicationContext as? Application + app?.registerActivityLifecycleCallbacks(object : Application.ActivityLifecycleCallbacks { + override fun onActivityStopped(activity: Activity) = flush() + override fun onActivityCreated(a: Activity, b: Bundle?) {} + override fun onActivityStarted(a: Activity) {} + override fun onActivityResumed(a: Activity) {} + override fun onActivityPaused(a: Activity) {} + override fun onActivitySaveInstanceState(a: Activity, b: Bundle) {} + override fun onActivityDestroyed(a: Activity) {} + }) + + timer = java.util.Timer("HarnessCoverageFlush", true).also { + it.scheduleAtFixedRate(object : java.util.TimerTask() { + override fun run() = flush() + }, 1000L, 1000L) + } + + Log.i(TAG, "pid=$pid, flushing to ${ecFile?.absolutePath}") + } + + fun flush() { + val file = ecFile ?: return + val agent = cachedAgent ?: return + try { + val bytes = agent.javaClass + .getMethod("getExecutionData", Boolean::class.javaPrimitiveType) + .invoke(agent, false) as ByteArray + file.writeBytes(bytes) + } catch (e: Exception) { + Log.w(TAG, "Failed to flush coverage data", e) + } + } +} diff --git a/packages/coverage-android/android/src/main/kotlin/com/harness/coverage/CoverageInitProvider.kt b/packages/coverage-android/android/src/main/kotlin/com/harness/coverage/CoverageInitProvider.kt new file mode 100644 index 00000000..59493a89 --- /dev/null +++ b/packages/coverage-android/android/src/main/kotlin/com/harness/coverage/CoverageInitProvider.kt @@ -0,0 +1,20 @@ +package com.harness.coverage + +import android.content.ContentProvider +import android.content.ContentValues +import android.database.Cursor +import android.net.Uri + +class CoverageInitProvider : ContentProvider() { + override fun onCreate(): Boolean { + val ctx = context ?: return true + CoverageHelper.setup(ctx) + return true + } + + override fun query(u: Uri, p: Array?, s: String?, a: Array?, o: String?): Cursor? = null + override fun getType(uri: Uri): String? = null + override fun insert(uri: Uri, values: ContentValues?): Uri? = null + override fun delete(uri: Uri, selection: String?, selectionArgs: Array?): Int = 0 + override fun update(uri: Uri, values: ContentValues?, selection: String?, selectionArgs: Array?): Int = 0 +} diff --git a/packages/coverage-android/android/src/main/resources/jacoco-agent.properties b/packages/coverage-android/android/src/main/resources/jacoco-agent.properties new file mode 100644 index 00000000..52c4b253 --- /dev/null +++ b/packages/coverage-android/android/src/main/resources/jacoco-agent.properties @@ -0,0 +1 @@ +output=none diff --git a/packages/coverage-android/package.json b/packages/coverage-android/package.json new file mode 100644 index 00000000..990ea1ee --- /dev/null +++ b/packages/coverage-android/package.json @@ -0,0 +1,37 @@ +{ + "name": "@react-native-harness/coverage-android", + "description": "Native Android code coverage support for React Native Harness.", + "version": "1.1.0", + "type": "module", + "exports": { + "./package.json": "./package.json", + ".": { + "development": "./src/index.ts", + "types": "./dist/index.d.ts", + "import": "./dist/index.js", + "default": "./dist/index.js" + } + }, + "files": [ + "src", + "dist", + "android", + "scripts", + "!**/__tests__", + "!**/__fixtures__", + "!**/__mocks__", + "!**/.*" + ], + "peerDependencies": { + "react-native": "*" + }, + "dependencies": { + "tslib": "^2.3.0" + }, + "devDependencies": { + "react-native": "*" + }, + "license": "MIT", + "homepage": "https://github.com/callstackincubator/react-native-harness", + "author": "React Native Harness contributors" +} diff --git a/packages/coverage-android/scripts/harness-coverage-init.gradle b/packages/coverage-android/scripts/harness-coverage-init.gradle new file mode 100644 index 00000000..f940853b --- /dev/null +++ b/packages/coverage-android/scripts/harness-coverage-init.gradle @@ -0,0 +1,174 @@ +/** + * React Native Harness — Android coverage init script. + * + * Applies JaCoCo offline instrumentation to specified Gradle modules so that + * coverage data can be collected from a running app (not just androidTest). + * + * Usage: + * ./gradlew assembleDebug \ + * --init-script node_modules/@react-native-harness/coverage-android/scripts/harness-coverage-init.gradle \ + * -PHarnessCoverageModules=:android + * + * The modules to instrument are read from the Gradle property + * `HarnessCoverageModules` (colon-prefixed, comma-separated, e.g. ":mylib,:other") + * or the environment variable `HARNESS_COVERAGE_MODULES`. + * + * For each target module this script: + * 1. Adds the JaCoCo agent runtime dependency (provides RT.getAgent()) + * 2. Saves original (uninstrumented) class files for report generation + * 3. Offline-instruments the compiled class files with JaCoCo probes + * 4. Copies jacococli.jar to the build output for later report generation + * 5. Injects the Harness coverage runtime (CoverageHelper + CoverageInitProvider) + * 6. Adds a COVERAGE_ENABLED BuildConfig field + * + * After the build, the following artifacts are available in each module's + * build/harness-coverage/ directory: + * - original-classes/ — uninstrumented class files needed for the report + * - jacococli.jar — JaCoCo CLI for merging .ec files and generating reports + */ + +def resolveCoverageModules() { + def prop = gradle.startParameter.projectProperties['HarnessCoverageModules'] + if (prop) return prop.split(',').collect { it.trim() } + + def env = System.getenv('HARNESS_COVERAGE_MODULES') + if (env) return env.split(',').collect { it.trim() } + + return [] +} + +def targetModules = resolveCoverageModules() +if (targetModules.isEmpty()) { + logger.warn('[HarnessCoverage] No modules specified — set -PHarnessCoverageModules=:mylib or HARNESS_COVERAGE_MODULES env var') + return +} + +logger.lifecycle("[HarnessCoverage] Will instrument modules: ${targetModules.join(', ')}") + +// Locate the coverage-android package directory (parent of scripts/) +def scriptDir = buildscript.sourceFile?.parentFile +def packageDir = scriptDir?.parentFile + +def jacocoVersion = '0.8.12' + +// Use allprojects + afterEvaluate so that source sets, dependencies, and +// BuildConfig fields are registered BEFORE the compile tasks finalize their +// inputs. gradle.projectsEvaluated is too late for source-set changes. +gradle.allprojects { project -> + if (!targetModules.contains(project.path)) return + + project.afterEvaluate { + logger.lifecycle("[HarnessCoverage] Configuring ${project.path}") + + def androidExt = project.extensions.findByName('android') + if (!androidExt) { + logger.warn("[HarnessCoverage] ${project.path} has no android extension, skipping") + return + } + + // --- Dependencies --- + project.configurations.maybeCreate('jacocoAnt') + def jacocoCliConf = project.configurations.maybeCreate('jacocoCli') + jacocoCliConf.transitive = false + + project.dependencies { + implementation "org.jacoco:org.jacoco.agent:${jacocoVersion}:runtime" + jacocoAnt "org.jacoco:org.jacoco.ant:${jacocoVersion}" + jacocoCli "org.jacoco:org.jacoco.cli:${jacocoVersion}:nodeps" + } + + // --- BuildConfig field --- + def namespace = androidExt.namespace + androidExt.defaultConfig { + buildConfigField "boolean", "COVERAGE_ENABLED", "true" + } + + // --- Inject coverage runtime sources --- + if (packageDir) { + def coverageSrcDir = new File(packageDir, 'android/src/main/kotlin') + def coverageResources = new File(packageDir, 'android/src/main/resources') + + androidExt.sourceSets { + debug { + kotlin.srcDirs += coverageSrcDir + resources.srcDirs += coverageResources + } + } + + // Generate a debug-overlay manifest that merges the ContentProvider + def generatedManifestDir = project.file("${project.buildDir}/generated/harness-coverage") + generatedManifestDir.mkdirs() + def generatedManifest = new File(generatedManifestDir, 'AndroidManifest.xml') + generatedManifest.text = """\ + + + + + +""" + androidExt.sourceSets.debug.manifest.srcFile generatedManifest + } + + // --- Offline instrumentation + stash build artifacts --- + def coverageDir = project.file("${project.buildDir}/harness-coverage") + + def instrumentClasses = { File classesDir, String label -> + if (!classesDir.exists()) return + + coverageDir.mkdirs() + + // Save original classes for report generation + def origDir = new File(coverageDir, "original-classes-${label}") + if (origDir.exists()) origDir.deleteDir() + ant.copy(todir: origDir) { + fileset(dir: classesDir) + } + + // Stash jacococli.jar for use at report time + def destJar = new File(coverageDir, 'jacococli.jar') + if (!destJar.exists()) { + def cliJar = project.configurations.jacocoCli.singleFile + ant.copy(file: cliJar, tofile: destJar) + logger.lifecycle("[HarnessCoverage] Stashed ${destJar}") + } + + // Instrument to temp dir, then replace originals + def instrumentedDir = project.file("${project.buildDir}/tmp/jacoco-instrumented-${label}") + if (instrumentedDir.exists()) instrumentedDir.deleteDir() + instrumentedDir.mkdirs() + + ant.taskdef( + name: 'instrument', + classname: 'org.jacoco.ant.InstrumentTask', + classpath: project.configurations.jacocoAnt.asPath + ) + ant.instrument(destdir: instrumentedDir) { + fileset(dir: classesDir, includes: '**/*.class') + } + + ant.copy(todir: classesDir, overwrite: true) { + fileset(dir: instrumentedDir) + } + + logger.lifecycle("[HarnessCoverage] Instrumented ${label} classes in ${classesDir}") + } + + // Hook into compile tasks. Use configureEach with name matching so it + // works with lazily-registered tasks (AGP + RN Gradle plugin register + // compile tasks after project evaluation, during task graph resolution). + project.tasks.configureEach { task -> + if (task.name == 'compileDebugKotlin') { + def kotlinClasses = project.file("${project.buildDir}/tmp/kotlin-classes/debug") + task.doLast { instrumentClasses(kotlinClasses, 'kotlin') } + } + if (task.name == 'compileDebugJavaWithJavac') { + def javaClasses = project.file("${project.buildDir}/intermediates/javac/debug/classes") + task.doLast { instrumentClasses(javaClasses, 'java') } + } + } + } +} diff --git a/packages/coverage-android/scripts/resolve-coverage-modules.mjs b/packages/coverage-android/scripts/resolve-coverage-modules.mjs new file mode 100644 index 00000000..abc9ddeb --- /dev/null +++ b/packages/coverage-android/scripts/resolve-coverage-modules.mjs @@ -0,0 +1,5 @@ +import { getConfig } from '@react-native-harness/config'; + +const { config } = await getConfig(process.cwd()); +const modules = config.coverage?.native?.android?.modules ?? []; +console.log(JSON.stringify(modules)); diff --git a/packages/coverage-android/src/index.ts b/packages/coverage-android/src/index.ts new file mode 100644 index 00000000..cb0ff5c3 --- /dev/null +++ b/packages/coverage-android/src/index.ts @@ -0,0 +1 @@ +export {}; diff --git a/packages/coverage-android/tsconfig.json b/packages/coverage-android/tsconfig.json new file mode 100644 index 00000000..af1b3657 --- /dev/null +++ b/packages/coverage-android/tsconfig.json @@ -0,0 +1,13 @@ +{ + "extends": "../../tsconfig.base.json", + "files": [], + "include": [], + "references": [ + { + "path": "../config" + }, + { + "path": "./tsconfig.lib.json" + } + ] +} diff --git a/packages/coverage-android/tsconfig.lib.json b/packages/coverage-android/tsconfig.lib.json new file mode 100644 index 00000000..385885be --- /dev/null +++ b/packages/coverage-android/tsconfig.lib.json @@ -0,0 +1,19 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "baseUrl": ".", + "rootDir": "src", + "outDir": "dist", + "tsBuildInfoFile": "dist/tsconfig.lib.tsbuildinfo", + "emitDeclarationOnly": false, + "forceConsistentCasingInFileNames": true, + "types": ["node"], + "lib": ["DOM", "ES2022"] + }, + "include": ["src/**/*.ts"], + "references": [ + { + "path": "../config/tsconfig.lib.json" + } + ] +} diff --git a/packages/coverage-ios/tsconfig.json b/packages/coverage-ios/tsconfig.json index c23e61c8..af1b3657 100644 --- a/packages/coverage-ios/tsconfig.json +++ b/packages/coverage-ios/tsconfig.json @@ -3,6 +3,9 @@ "files": [], "include": [], "references": [ + { + "path": "../config" + }, { "path": "./tsconfig.lib.json" } diff --git a/packages/coverage-ios/tsconfig.lib.json b/packages/coverage-ios/tsconfig.lib.json index 7370b55e..385885be 100644 --- a/packages/coverage-ios/tsconfig.lib.json +++ b/packages/coverage-ios/tsconfig.lib.json @@ -10,5 +10,10 @@ "types": ["node"], "lib": ["DOM", "ES2022"] }, - "include": ["src/**/*.ts"] + "include": ["src/**/*.ts"], + "references": [ + { + "path": "../config/tsconfig.lib.json" + } + ] } diff --git a/packages/jest/src/harness-session.ts b/packages/jest/src/harness-session.ts index a19b2387..7443ce40 100644 --- a/packages/jest/src/harness-session.ts +++ b/packages/jest/src/harness-session.ts @@ -558,19 +558,34 @@ export const createHarnessSession = async ( bridge.off('disconnected', onDisconnected); bridge.off('event', bridgeEventListener); - const nativeCoverageConfig = runtimeConfig.coverage?.native?.ios; - if (nativeCoverageConfig?.pods?.length && platformInstance.collectNativeCoverage) { - try { - await platformInstance.stopApp(); - const lcovPath = await platformInstance.collectNativeCoverage({ - pods: nativeCoverageConfig.pods, - outputDir: projectRoot, - }); - if (lcovPath) { - logNativeCoverageCollected(lcovPath); + if (platformInstance.collectNativeCoverage) { + const isAndroid = platform.platformId === 'android'; + const iosPods = runtimeConfig.coverage?.native?.ios?.pods; + const androidModules = runtimeConfig.coverage?.native?.android?.modules; + + const coverageOptions = isAndroid + ? androidModules?.length + ? { modules: androidModules, outputDir: projectRoot } + : null + : iosPods?.length + ? { pods: iosPods, outputDir: projectRoot } + : null; + + if (coverageOptions) { + try { + await platformInstance.stopApp(); + if (isAndroid) { + // JaCoCo flush timer runs every 1s; wait for final write + await new Promise((r) => setTimeout(r, 2000)); + } + const lcovPath = + await platformInstance.collectNativeCoverage(coverageOptions); + if (lcovPath) { + logNativeCoverageCollected(lcovPath); + } + } catch (error) { + sessionLogger.warn('failed to collect native coverage: %s', error); } - } catch (error) { - sessionLogger.warn('failed to collect native coverage: %s', error); } } diff --git a/packages/platform-android/src/coverage-collector.ts b/packages/platform-android/src/coverage-collector.ts new file mode 100644 index 00000000..75b7ebb3 --- /dev/null +++ b/packages/platform-android/src/coverage-collector.ts @@ -0,0 +1,380 @@ +import { spawn, logger } from '@react-native-harness/tools'; +import { getAdbBinaryPath } from './environment.js'; +import fs from 'node:fs'; +import path from 'node:path'; + +const coverageLogger = logger.child('android-coverage'); + +export const pullEcFiles = async ( + adbId: string, + bundleId: string, + localDir: string +): Promise => { + fs.mkdirSync(localDir, { recursive: true }); + + const adb = getAdbBinaryPath(); + const tmpDir = `/data/local/tmp/harness-coverage-${Date.now()}`; + + try { + // List .ec files in app internal storage + const { stdout: listing } = await spawn(adb, [ + '-s', + adbId, + 'shell', + 'run-as', + bundleId, + 'ls', + 'files/', + ]); + + const ecFileNames = listing + .split('\n') + .map((f) => f.trim()) + .filter((f) => f.endsWith('.ec')); + + if (ecFileNames.length === 0) { + return []; + } + + // Copy files to a world-readable temp dir so `adb pull` can access them. + // `run-as` files are in the app's private directory, which `adb pull` + // cannot read directly. + await spawn(adb, ['-s', adbId, 'shell', 'mkdir', '-p', tmpDir]); + + for (const ecFile of ecFileNames) { + try { + await spawn(adb, [ + '-s', + adbId, + 'shell', + 'run-as', + bundleId, + 'cp', + `files/${ecFile}`, + `${tmpDir}/${ecFile}`, + ]); + } catch (e) { + coverageLogger.debug('Failed to copy %s to tmp: %s', ecFile, e); + } + } + + // Pull all .ec files from the temp dir using adb pull (handles binary correctly) + await spawn(adb, ['-s', adbId, 'pull', tmpDir + '/.', localDir]); + } catch (e) { + coverageLogger.debug('Failed to pull .ec files: %s', e); + } finally { + // Clean up the temp dir on device + try { + await spawn(adb, ['-s', adbId, 'shell', 'rm', '-rf', tmpDir]); + } catch { + // best-effort cleanup + } + } + + return fs + .readdirSync(localDir) + .filter((f) => f.endsWith('.ec')) + .map((f) => path.join(localDir, f)); +}; + +/** + * Finds the harness-coverage build artifacts for a module. + * The Gradle init script saves these to /build/harness-coverage/: + * - original-classes-kotlin/ (uninstrumented Kotlin class files) + * - original-classes-java/ (uninstrumented Java class files) + * - jacococli.jar (JaCoCo CLI) + */ +const findCoverageArtifacts = ( + projectRoot: string, + modules: string[] +): { + jacococliJar: string | null; + classesDirs: string[]; + sourceDirs: string[]; +} => { + let jacococliJar: string | null = null; + const classesDirs: string[] = []; + const sourceDirs: string[] = []; + + for (const mod of modules) { + const modDir = mod.replace(/^:/, ''); + const coverageDir = path.join( + projectRoot, + modDir, + 'build', + 'harness-coverage' + ); + + let foundClasses = false; + for (const suffix of ['kotlin', 'java']) { + const classesDir = path.join(coverageDir, `original-classes-${suffix}`); + if (fs.existsSync(classesDir)) { + classesDirs.push(classesDir); + foundClasses = true; + } + } + if (!foundClasses) { + coverageLogger.warn( + '[coverage] Original class files not found in %s — was the app built with the coverage init script?', + coverageDir + ); + } + + if (!jacococliJar) { + const jar = path.join(coverageDir, 'jacococli.jar'); + if (fs.existsSync(jar)) { + jacococliJar = jar; + } + } + + for (const srcPath of [ + path.join(projectRoot, modDir, 'src', 'main', 'java'), + path.join(projectRoot, modDir, 'src', 'main', 'kotlin'), + ]) { + if (fs.existsSync(srcPath)) { + sourceDirs.push(srcPath); + } + } + } + + return { jacococliJar, classesDirs, sourceDirs }; +}; + +const mergeEcFiles = async ( + jacococliJar: string, + ecFiles: string[], + outputPath: string +): Promise => { + await spawn('java', [ + '-jar', + jacococliJar, + 'merge', + ...ecFiles, + '--destfile', + outputPath, + ]); +}; + +const generateXmlReport = async (options: { + jacococliJar: string; + execFile: string; + classesDirs: string[]; + sourceDirs: string[]; + outputPath: string; +}): Promise => { + const { jacococliJar, execFile, classesDirs, sourceDirs, outputPath } = + options; + + const args = ['-jar', jacococliJar, 'report', execFile]; + + for (const dir of classesDirs) { + args.push('--classfiles', dir); + } + for (const dir of sourceDirs) { + args.push('--sourcefiles', dir); + } + args.push('--xml', outputPath); + + await spawn('java', args); +}; + +const getAttr = (tag: string, name: string): string => { + const match = tag.match(new RegExp(`${name}="([^"]*)"`)); + return match?.[1] ?? ''; +}; + +/** + * Resolves a package-relative source path (e.g. "com/example/Foo.kt") to an + * absolute path by checking which source directory contains the file. + */ +const resolveSourcePath = ( + relativePath: string, + sourceDirs: string[] +): string => { + for (const srcDir of sourceDirs) { + const candidate = path.join(srcDir, relativePath); + if (fs.existsSync(candidate)) { + return candidate; + } + } + return relativePath; +}; + +/** + * Converts a JaCoCo XML report to lcov format. + * + * JaCoCo XML structure: + * + * + * + * + * + * + * + */ +export const convertJacocoXmlToLcov = ( + xmlPath: string, + lcovPath: string, + sourceDirs: string[] = [] +): void => { + const xml = fs.readFileSync(xmlPath, 'utf-8'); + const output: string[] = []; + + let currentPackage = ''; + let fileLines: Array<{ nr: number; hits: number }> = []; + let filePath = ''; + + const tagRegex = /<(\/?)(\w+)([^>]*?)(\/?)>/g; + let match; + + while ((match = tagRegex.exec(xml)) !== null) { + const isClosing = match[1] === '/'; + const tagName = match[2]; + const attrs = match[3]; + const isSelfClosing = match[4] === '/'; + + if (!isClosing) { + if (tagName === 'package') { + currentPackage = getAttr(attrs, 'name'); + } else if (tagName === 'sourcefile') { + const fileName = getAttr(attrs, 'name'); + filePath = currentPackage + ? `${currentPackage}/${fileName}` + : fileName; + fileLines = []; + } else if (tagName === 'line' && filePath) { + const nr = parseInt(getAttr(attrs, 'nr') || '0', 10); + const ci = parseInt(getAttr(attrs, 'ci') || '0', 10); + const mi = parseInt(getAttr(attrs, 'mi') || '0', 10); + if (nr > 0 && (ci > 0 || mi > 0)) { + fileLines.push({ nr, hits: ci }); + } + } + } + + if (isClosing || isSelfClosing) { + if (tagName === 'sourcefile' && filePath && fileLines.length > 0) { + const resolvedPath = resolveSourcePath(filePath, sourceDirs); + output.push(`SF:${resolvedPath}`); + for (const line of fileLines) { + output.push(`DA:${line.nr},${line.hits}`); + } + output.push(`LF:${fileLines.length}`); + const hitLines = fileLines.filter((l) => l.hits > 0).length; + output.push(`LH:${hitLines}`); + output.push('end_of_record'); + filePath = ''; + fileLines = []; + } else if (tagName === 'package') { + currentPackage = ''; + } + } + } + + fs.writeFileSync(lcovPath, output.join('\n') + '\n'); +}; + +export const cleanEcFilesOnDevice = async ( + adbId: string, + bundleId: string +): Promise => { + const adb = getAdbBinaryPath(); + try { + await spawn(adb, [ + '-s', + adbId, + 'shell', + 'run-as', + bundleId, + 'sh', + '-c', + 'rm -f files/coverage-*.ec', + ]); + } catch { + coverageLogger.debug('Failed to clean .ec files from device'); + } +}; + +export type CollectAndroidNativeCoverageOptions = { + adbId: string; + bundleId: string; + modules: string[]; + outputDir: string; +}; + +export const collectAndroidNativeCoverage = async ( + options: CollectAndroidNativeCoverageOptions +): Promise => { + const { adbId, bundleId, modules, outputDir } = options; + + coverageLogger.debug('[coverage] Collecting native Android coverage', { + adbId, + bundleId, + modules, + }); + + // Pull .ec files from device + const ecDir = path.join(outputDir, '.harness-coverage-ec'); + const ecFiles = await pullEcFiles(adbId, bundleId, ecDir); + + if (ecFiles.length === 0) { + coverageLogger.debug('[coverage] No .ec files found on device'); + return null; + } + + coverageLogger.debug(`[coverage] Found ${ecFiles.length} .ec file(s)`); + + // Find build artifacts (jacococli.jar + original classes) from the Android + // project. For RN apps the Android project is typically at outputDir/android/. + let androidProjectDir = outputDir; + const androidSubdir = path.join(outputDir, 'android'); + if (fs.existsSync(androidSubdir)) { + androidProjectDir = androidSubdir; + } + + const { jacococliJar, classesDirs, sourceDirs } = findCoverageArtifacts( + androidProjectDir, + modules + ); + + if (!jacococliJar) { + coverageLogger.warn( + '[coverage] jacococli.jar not found in build output — was the app built with the coverage init script?' + ); + fs.rmSync(ecDir, { recursive: true, force: true }); + return null; + } + + if (classesDirs.length === 0) { + coverageLogger.warn( + '[coverage] No original class files found — cannot generate report' + ); + fs.rmSync(ecDir, { recursive: true, force: true }); + return null; + } + + // Merge .ec files + const mergedEc = path.join(outputDir, 'native-coverage.exec'); + await mergeEcFiles(jacococliJar, ecFiles, mergedEc); + + // Generate XML report + const xmlPath = path.join(outputDir, 'native-coverage.xml'); + await generateXmlReport({ + jacococliJar, + execFile: mergedEc, + classesDirs, + sourceDirs, + outputPath: xmlPath, + }); + + // Convert to lcov + const lcovPath = path.join(outputDir, 'native-coverage.lcov'); + convertJacocoXmlToLcov(xmlPath, lcovPath, sourceDirs); + + // Clean up + fs.rmSync(ecDir, { recursive: true, force: true }); + await cleanEcFilesOnDevice(adbId, bundleId); + + coverageLogger.debug(`[coverage] Native coverage written to: ${lcovPath}`); + return lcovPath; +}; diff --git a/packages/platform-android/src/instance.ts b/packages/platform-android/src/instance.ts index d28eb177..cd10eeea 100644 --- a/packages/platform-android/src/instance.ts +++ b/packages/platform-android/src/instance.ts @@ -1,5 +1,6 @@ import { AppNotInstalledError, + type CollectNativeCoverageOptions, CreateAppMonitorOptions, DeviceNotFoundError, type HarnessPlatformInitOptions, @@ -329,6 +330,17 @@ export const getAndroidEmulatorPlatformInstance = async ( crashArtifactWriter: options?.crashArtifactWriter, }); }, + collectNativeCoverage: async (options: CollectNativeCoverageOptions) => { + const { collectAndroidNativeCoverage } = await import( + './coverage-collector.js' + ); + return collectAndroidNativeCoverage({ + adbId, + bundleId: config.bundleId, + modules: options.modules ?? [], + outputDir: options.outputDir, + }); + }, }; }; @@ -404,5 +416,16 @@ export const getAndroidPhysicalDevicePlatformInstance = async ( crashArtifactWriter: options?.crashArtifactWriter, }); }, + collectNativeCoverage: async (options: CollectNativeCoverageOptions) => { + const { collectAndroidNativeCoverage } = await import( + './coverage-collector.js' + ); + return collectAndroidNativeCoverage({ + adbId, + bundleId: config.bundleId, + modules: options.modules ?? [], + outputDir: options.outputDir, + }); + }, }; }; diff --git a/packages/platform-ios/src/instance.ts b/packages/platform-ios/src/instance.ts index 36c62cd1..543cbb1c 100644 --- a/packages/platform-ios/src/instance.ts +++ b/packages/platform-ios/src/instance.ts @@ -203,7 +203,7 @@ export const getAppleSimulatorPlatformInstance = async ( return await collectNativeCoverage({ udid, bundleId: config.bundleId, - pods: options.pods, + pods: options.pods ?? [], outputDir: options.outputDir, }); }, diff --git a/packages/platforms/src/types.ts b/packages/platforms/src/types.ts index f4d16819..ac40c2eb 100644 --- a/packages/platforms/src/types.ts +++ b/packages/platforms/src/types.ts @@ -99,8 +99,9 @@ export type AppLaunchOptions = | VegaAppLaunchOptions; export type CollectNativeCoverageOptions = { - pods: string[]; outputDir: string; + pods?: string[]; + modules?: string[]; }; export type HarnessPlatformRunner = { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 50b36526..4d321a31 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -344,6 +344,16 @@ importers: specifier: ^3.25.67 version: 3.25.67 + packages/coverage-android: + dependencies: + tslib: + specifier: ^2.3.0 + version: 2.8.1 + devDependencies: + react-native: + specifier: '*' + version: 0.82.1(@babel/core@7.27.4)(@react-native-community/cli@20.0.0(typescript@5.9.3))(@react-native/metro-config@0.82.1)(@types/react@19.1.13)(react@19.2.3) + packages/coverage-ios: dependencies: tslib: diff --git a/tsconfig.json b/tsconfig.json index 4d6eb235..f8ebf30f 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -62,6 +62,9 @@ }, { "path": "./packages/coverage-ios" + }, + { + "path": "./packages/coverage-android" } ] } diff --git a/website/src/docs/getting-started/configuration.mdx b/website/src/docs/getting-started/configuration.mdx index f2003098..cc1e67a8 100644 --- a/website/src/docs/getting-started/configuration.mdx +++ b/website/src/docs/getting-started/configuration.mdx @@ -108,6 +108,7 @@ For Expo projects, the `entryPoint` should be set to the path specified in the ` | `coverage` | Coverage configuration object. | | `coverage.root` | Root directory for coverage instrumentation (default: `process.cwd()`). | | `coverage.native.ios.pods` | Experimental list of CocoaPods target names to instrument for iOS native coverage. | +| `coverage.native.android.modules` | Experimental list of Gradle module paths to instrument for Android native coverage (e.g. `[":mylib"]`). | | `forwardClientLogs` | Forward console logs from the app to the terminal (default: `false`). | | `unstable__enableMetroCache` | Enable Metro transformation cache under `.harness/metro-cache` and log when reusing it (default: `false`). | diff --git a/website/src/docs/guides/native-coverage.mdx b/website/src/docs/guides/native-coverage.mdx index 5ed6e559..d3ac114d 100644 --- a/website/src/docs/guides/native-coverage.mdx +++ b/website/src/docs/guides/native-coverage.mdx @@ -5,21 +5,21 @@ import { PackageManagerTabs } from '@theme'; React Native Harness provides experimental support for collecting native coverage during test runs. :::warning Experimental -`@react-native-harness/coverage-ios` is an **experimental** feature. Expect rough edges and API changes while the integration matures. +Native coverage packages are **experimental**. Expect rough edges and API changes while the integration matures. ::: -Today, native coverage support is available for **iOS simulators only**. Physical iOS devices are not supported yet. Android support is planned and this guide will expand as it lands. +Native coverage support is available for **iOS simulators** and **Android emulators/devices**. ## What you get - Coverage instrumentation for supported native dependencies -- Automatic `.profraw` collection from the app sandbox +- Automatic coverage data collection from the app - `native-coverage.lcov` output after the test run finishes - A way to measure native code exercised by Harness tests, alongside JavaScript coverage -## Installation +## iOS -Current package: +### Installation @@ -28,9 +28,7 @@ After installation: 1. Run your iOS pod install step. 2. Rebuild the app. -## Configuration - -Current iOS configuration: +### Configuration Add the pods you want to instrument in `rn-harness.config.mjs`: @@ -57,11 +55,7 @@ export default { The `pods` array should contain CocoaPods target names that you want Harness to instrument for coverage. -## Running tests - -Current iOS command: - -Run Harness with coverage enabled: +### Running tests ```bash react-native-harness --coverage --harnessRunner ios @@ -69,14 +63,14 @@ react-native-harness --coverage --harnessRunner ios Harness will stop the app before cleanup, collect generated `.profraw` files, merge them with `llvm-profdata`, and export LCOV with `llvm-cov`. -## Output +### Output When native coverage is collected successfully, Harness writes these files to the project root: - `native-coverage.profdata` - `native-coverage.lcov` -## Requirements +### Requirements - macOS with Xcode installed - An iOS runner configured with `@react-native-harness/platform-apple` @@ -84,9 +78,82 @@ When native coverage is collected successfully, Harness writes these files to th - iOS Simulator - Debug app build -## Limitations +### Limitations -- iOS simulator support is available today; physical iOS devices and Android are not supported yet +- iOS simulator only; physical iOS devices are not supported yet - Current implementation targets pod-based native code -- The feature is experimental and may change without much notice - If LCOV source filtering fails for pod paths, Harness falls back to exporting broader coverage data + +## Android + +### Installation + + + +After installation, rebuild the app with the coverage init script (see below). + +### Build with coverage + +The app must be built with JaCoCo offline instrumentation using the provided Gradle init script: + +```bash +cd android +./gradlew assembleDebug \ + --init-script ../node_modules/@react-native-harness/coverage-android/scripts/harness-coverage-init.gradle \ + -PHarnessCoverageModules=:mylib +cd .. +``` + +Replace `:mylib` with your library's Gradle module path. Multiple modules can be specified with commas: `-PHarnessCoverageModules=:moduleA,:moduleB`. + +### Configuration + +Add the modules you want to instrument in `rn-harness.config.mjs`: + +```javascript +import { androidPlatform, androidEmulator } from '@react-native-harness/platform-android'; + +export default { + runners: [ + androidPlatform({ + name: 'android', + device: androidEmulator('Pixel_8_API_35'), + bundleId: 'com.example.app', + }), + ], + coverage: { + native: { + android: { + modules: [':mylib'], + }, + }, + }, +}; +``` + +The `modules` array must match the module paths passed to the init script during the build. + +### Running tests + +```bash +react-native-harness --coverage --harnessRunner android +``` + +Harness will stop the app, pull `.ec` execution data files from the device, merge them with JaCoCo CLI, and convert the report to LCOV. + +### Output + +When native coverage is collected successfully, Harness writes `native-coverage.lcov` to the project root. + +### Requirements + +- Android SDK with an emulator or physical device +- Java 11+ +- An Android runner configured with `@react-native-harness/platform-android` +- Debug app build using the coverage init script +- `@react-native-harness/coverage-android` installed + +### Limitations + +- Requires building with the Gradle init script (`--init-script`) +- Build and test environments must share access to the build output (original class files + `jacococli.jar` in `/build/harness-coverage/`)