feat: suggest the developers behind your stack when signing up

KNOWLEDGE_BASE.md

@@ -148,9 +148,11 @@ Sources: [badge definitions](app/Enums/Achievement.php), [unlock conditions and
 
 Follow or unfollow from a developer's profile or a follow control elsewhere in the app. Following puts their commits in your Following log. Following or unfollowing from a log updates the controls, counts, and live-following sidebar without navigating away: Latest and Trending keep their loaded commits and ranking, while Following updates its commits and empty-state suggestions. You cannot follow yourself or follow across a block. GitHub follows are imported once: connections to developers already on /dev/null are added in both directions, with up to 1,000 GitHub connections checked each way. This is not continuous synchronization, so unfollowing someone on /dev/null sticks.
 
-**Who to follow** suggests developers in the side column or within the log on smaller screens, each with the first line of their bio and why they are suggested (how many people you follow follow them, or their follower count). Follow from the icon next to each suggestion; for someone who already follows you, it follows back. The X icon marks a suggestion **Not interested**: that developer is never suggested to you again, without muting them, and the next suggestion takes their place. Refresh for different suggestions, or use **… → Don't show again** to hide the block, with an immediate **Undo** option. Show or hide these suggestions in Settings → Appearance. Suggested developers must have a visible top-level commit. On profiles, suggestions prioritize the people that developer follows; on commit pages, they prioritize visible thread participants. Regular suggestions fill any remaining places. You can also follow back from the notifications list.
+**Who to follow** suggests developers in the side column or within the log on smaller screens, each with the first line of their bio and why they are suggested (behind your stack, how many people you follow follow them, or their follower count). Follow from the icon next to each suggestion; for someone who already follows you, it follows back. The X icon marks a suggestion **Not interested**: that developer is never suggested to you again, without muting them, and the next suggestion takes their place. Refresh for different suggestions, or use **… → Don't show again** to hide the block, with an immediate **Undo** option. Show or hide these suggestions in Settings → Appearance. Suggested developers must have a visible top-level commit. On profiles, suggestions prioritize the people that developer follows; on commit pages, they prioritize visible thread participants. Regular suggestions fill any remaining places. You can also follow back from the notifications list.
 
-Sources: [follow import](app/Actions/CreateUserGitHubFollows.php), [suggestions](resources/js/components/who-to-follow.tsx), [suggestion eligibility](app/Queries/SuggestedUsersQuery.php), [appearance settings](resources/js/pages/appearance/update.tsx), [notifications page](resources/js/pages/notification/index.tsx).
+For a week after you sign up, the developers behind your stack are suggested during onboarding and in Who to follow, ahead of the other people you might know and alongside a few popular developers: of your 10 most recently pushed public repositories, /dev/null reads the `composer.json` and `package.json` at the root of the ones that aren't forks, takes the 20 Composer and npm packages used by the most of them (leaving out `@types/*`), and suggests whoever among the top five GitHub contributors to each package's repository is on /dev/null. When those packages come from fewer than 20 repositories, the repositories you starred most recently, other than your own, fill the rest, so a stack that lives in an organization's repositories or outside Composer and npm still finds its people. On profiles and commit pages, they are among the regular suggestions that fill the places the page's own suggestions leave. The lookups stop after 20 seconds, keeping whatever was found by then. They are suggested rather than followed for you, since anyone you follow can message you in Remotes. Only who to suggest is kept, for that week, and nothing about your repositories.
+
+Sources: [follow import](app/Actions/CreateUserGitHubFollows.php), [the developers behind your stack](app/Actions/CreateStackSuggestions.php), [suggestions](resources/js/components/who-to-follow.tsx), [suggestion eligibility](app/Queries/SuggestedUsersQuery.php), [appearance settings](resources/js/pages/appearance/update.tsx), [notifications page](resources/js/pages/notification/index.tsx).
 
 ### Following, Latest, and Trending logs
 

app/Actions/CreateStackSuggestions.php

@@ -0,0 +1,42 @@
+<?php
+
+declare(strict_types=1);
+
+namespace App\Actions;
+
+use App\Models\User;
+use App\Queries\DependencyContributorsQuery;
+use Illuminate\Support\Facades\Cache;
+
+/**
+ * Remembers who is behind a user's stack so they are suggested to follow, rather than followed for them: a follow
+ * lets someone message you, and that should be your choice.
+ */
+final readonly class CreateStackSuggestions
+{
+    /**
+     * Long enough to cover someone's first days, after which the usual suggestions take over.
+     */
+    private const int Days = 7;
+
+    public function __construct(private DependencyContributorsQuery $dependencyContributors)
+    {
+        //
+    }
+
+    public static function key(User $user): string
+    {
+        return "stack-suggestions:{$user->id}";
+    }
+
+    public function handle(User $user): void
+    {
+        $ids = User::query()
+            ->whereIn('github_id', $this->dependencyContributors->get($user->handle))
+            ->whereKeyNot($user->id)
+            ->pluck('id')
+            ->all();
+
+        Cache::put(self::key($user), $ids, now()->addDays(self::Days));
+    }
+}

app/Actions/DeleteUser.php

@@ -70,6 +70,8 @@ public function handle(User $user): void
                 Cache::forget($key);
             }
 
+            Cache::forget(CreateStackSuggestions::key($user));
+
             return $imagePaths;
         });
 

app/Http/Controllers/SessionController.php

@@ -8,6 +8,7 @@
 use App\Actions\DeletePushSubscription;
 use App\Actions\UpdateOrCreateUser;
 use App\Jobs\ImportGitHubFollowsJob;
+use App\Jobs\SuggestDependencyContributorsJob;
 use App\Jobs\SyncGitHubSshKeysJob;
 use App\Models\User;
 use App\Services\GitHubAccountScreening;
@@ -59,6 +60,10 @@ public function store(Request $request, UpdateOrCreateUser $action, CreateTomFri
         dispatch(new SyncGitHubSshKeysJob($user));
         dispatch(new ImportGitHubFollowsJob($user, notify: $user->wasRecentlyCreated));
 
+        if ($user->wasRecentlyCreated) {
+            dispatch(new SuggestDependencyContributorsJob($user));
+        }
+
         Auth::login($user, remember: true);
 
         $request->session()->regenerate();

app/Jobs/SuggestDependencyContributorsJob.php

@@ -0,0 +1,31 @@
+<?php
+
+declare(strict_types=1);
+
+namespace App\Jobs;
+
+use App\Actions\CreateStackSuggestions;
+use App\Models\User;
+use Illuminate\Contracts\Queue\ShouldQueue;
+use Illuminate\Foundation\Queue\Queueable;
+use Illuminate\Queue\Attributes\Backoff;
+use Illuminate\Queue\Attributes\Timeout;
+use Illuminate\Queue\Attributes\Tries;
+
+#[Backoff([10, 60])]
+#[Timeout(120)]
+#[Tries(3)]
+final class SuggestDependencyContributorsJob implements ShouldQueue
+{
+    use Queueable;
+
+    public function __construct(public User $user)
+    {
+        //
+    }
+
+    public function handle(CreateStackSuggestions $createStackSuggestions): void
+    {
+        $createStackSuggestions->handle($this->user);
+    }
+}

app/Queries/DependencyContributorsQuery.php

@@ -0,0 +1,250 @@
+<?php
+
+declare(strict_types=1);
+
+namespace App\Queries;
+
+use App\Services\GitHub;
+use Illuminate\Http\Client\ConnectionException;
+use Illuminate\Http\Client\HttpClientException;
+use Illuminate\Http\Client\PendingRequest;
+use Illuminate\Http\Client\RequestException;
+use Illuminate\Support\Facades\Cache;
+use Illuminate\Support\Facades\Http;
+use Illuminate\Support\Fluent;
+use Illuminate\Support\Str;
+
+final readonly class DependencyContributorsQuery
+{
+    /**
+     * The most recently pushed repositories read, so a prolific account can't use up the rate limit on its own.
+     */
+    private const int Repositories = 10;
+
+    /**
+     * The packages looked up, the ones the most repositories depend on first, and the most repositories whose
+     * contributors are asked about.
+     */
+    private const int Packages = 20;
+
+    /**
+     * The top contributors to a package's repository who count as the people behind it.
+     */
+    private const int Contributors = 5;
+
+    /**
+     * How long reading and looking up may take altogether. The people behind someone's stack are suggested while they
+     * find their feet, so whatever isn't looked up by then is left out rather than waited for.
+     */
+    private const int Seconds = 20;
+
+    /**
+     * A GitHub repository as registries list one: an https, git or ssh URL, a github: one, or npm's owner/name shorthand.
+     * Neither part can start with a dot, so a listed repository can't point anywhere else on GitHub's API.
+     */
+    private const string RepositoryUrl = '~^(?:(?:git\+)?(?:https?|git|ssh)://(?:git@)?(?:www\.)?github\.com/|git@github\.com:|github:)?(?<repository>[\w-][\w.-]{0,38}/[\w-][\w.-]{0,99}?)(?:\.git)?/?(?:#.*)?$~i';
+
+    public function __construct(private GitHub $github)
+    {
+        //
+    }
+
+    /**
+     * The GitHub IDs of the people behind a user's stack: the top contributors to the repositories of the Composer and
+     * npm packages their own public repositories depend on, read from the composer.json and package.json at their root.
+     * When that finds fewer repositories than it may ask about, the ones they starred most recently fill the rest.
+     *
+     * @return array<int, string>
+     */
+    public function get(string $login): array
+    {
+        $deadline = now()->addSeconds(self::Seconds);
+        $inTime = fn (): bool => now()->isBefore($deadline);
+
+        $repositories = $this->github->request()
+            ->get("users/{$login}/repos", ['type' => 'owner', 'sort' => 'pushed', 'per_page' => self::Repositories])
+            ->throw()
+            ->collect()
+            ->mapInto(Fluent::class)
+            ->reject(fn (Fluent $repository): bool => $repository->boolean('fork'))
+            ->flatMap(fn (Fluent $repository): array => $inTime() ? $this->dependencies($repository->string('full_name')->value()) : [])
+            ->countBy()
+            ->sortDesc()
+            ->take(self::Packages)
+            ->keys()
+            ->map(fn (string $package): ?string => $inTime() ? $this->repository($package) : null)
+            ->filter()
+            ->unique();
+
+        return $repositories
+            ->concat($repositories->count() < self::Packages && $inTime() ? $this->starred($login) : [])
+            ->unique()
+            ->take(self::Packages)
+            ->flatMap(fn (string $repository): array => $inTime() ? $this->contributors($repository) : [])
+            ->unique()
+            ->values()
+            ->all();
+    }
+
+    /**
+     * The packages a repository depends on, in and out of development, as "composer:vendor/name" and "npm:name".
+     *
+     * @return list<string>
+     */
+    private function dependencies(string $repository): array
+    {
+        return [
+            ...$this->packages('composer', $this->manifest($repository, 'composer.json'), ['require', 'require-dev']),
+            ...$this->packages('npm', $this->manifest($repository, 'package.json'), ['dependencies', 'devDependencies']),
+        ];
+    }
+
+    /**
+     * The package names in the given sections of a manifest. Platform requirements such as php and ext-json have no
+     * vendor, so Composer's naming rules leave them out along with anything that isn't a package name. Types from @types
+     * are left out too: DefinitelyTyped's contributors wrote them, not the people behind the packages they describe.
+     *
+     * @param  'composer'|'npm'  $registry
+     * @param  Fluent<array-key, mixed>  $manifest
+     * @param  list<string>  $sections
+     * @return array<int, string>
+     */
+    private function packages(string $registry, Fluent $manifest, array $sections): array
+    {
+        $pattern = match ($registry) {
+            'composer' => '#^[a-z0-9]([_.-]?[a-z0-9]+)*/[a-z0-9](([_.]|-{1,2})?[a-z0-9]+)*$#',
+            'npm' => '#^(@[a-z0-9-~][a-z0-9-._~]*/)?[a-z0-9-~][a-z0-9-._~]*$#',
+        };
+
+        return collect($sections)
+            ->flatMap(fn (string $section): array => array_keys($manifest->array($section)))
+            ->filter(fn (int|string $name): bool => preg_match($pattern, (string) $name) === 1)
+            ->reject(fn (int|string $name): bool => Str::startsWith((string) $name, '@types/'))
+            ->map(fn (int|string $name): string => "{$registry}:{$name}")
+            ->unique()
+            ->values()
+            ->all();
+    }
+
+    /**
+     * A manifest at the root of a repository's default branch, empty when there is none. Only the first 100 KB are
+     * asked for: a real manifest is far smaller, and a bigger one stops being valid JSON once cut, so it is skipped.
+     * When GitHub throttles the reads, the lookup fails so it is tried again later, instead of quietly finding nobody.
+     *
+     * @return Fluent<array-key, mixed>
+     */
+    private function manifest(string $repository, string $file): Fluent
+    {
+        try {
+            $manifest = $this->http()
+                ->withHeaders(['Range' => 'bytes=0-102399'])
+                ->get("https://raw.githubusercontent.com/{$repository}/HEAD/{$file}")
+                ->throwIfStatus(429)
+                ->json();
+        } catch (ConnectionException) {
+            return new Fluent;
+        }
+
+        return new Fluent(is_array($manifest) ? $manifest : []);
+    }
+
+    /**
+     * The GitHub repository a package is developed in, like "laravel/framework", as its registry lists it. A package
+     * without one is remembered too, so it doesn't cost the next person a request; a failed request isn't, so it's retried.
+     * When the registry throttles it, the lookup fails so it is tried again later, instead of quietly finding nobody.
+     */
+    private function repository(string $package): ?string
+    {
+        try {
+            $repository = Cache::remember('dependency-repository:'.hash('sha256', $package), 86400, function () use ($package): string|false {
+                $name = Str::after($package, ':');
+
+                if (Str::startsWith($package, 'composer:')) {
+                    $url = data_get($this->registry("https://repo.packagist.org/p2/{$name}.json"), ['packages', $name, 0, 'source', 'url']);
+                } else {
+                    $document = $this->registry('https://registry.npmjs.org/'.str_replace('/', '%2f', $name).'/latest');
+
+                    // npm lists the repository either as an object with a URL or as the URL on its own.
+                    $url = data_get($document, 'repository.url', data_get($document, 'repository'));
+                }
+
+                return is_string($url) && preg_match(self::RepositoryUrl, $url, $matches) === 1 ? $matches['repository'] : false;
+            });
+        } catch (HttpClientException $httpClientException) {
+            throw_if($httpClientException instanceof RequestException && $httpClientException->response->tooManyRequests(), $httpClientException);
+
+            return null;
+        }
+
+        return $repository === false ? null : $repository;
+    }
+
+    /**
+     * The repositories a user starred most recently, other than their own. They speak for people whose own repositories
+     * say little about what they use: when their work lives in an organization's repositories, or outside Composer and npm.
+     *
+     * @return array<int, string>
+     */
+    private function starred(string $login): array
+    {
+        return $this->github->request()
+            ->get("users/{$login}/starred", ['per_page' => self::Packages])
+            ->throw()
+            ->collect()
+            ->mapInto(Fluent::class)
+            ->reject(fn (Fluent $repository): bool => strcasecmp($repository->string('owner.login')->value(), $login) === 0)
+            ->map(fn (Fluent $repository): string => $repository->string('full_name')->value())
+            ->values()
+            ->all();
+    }
+
+    /**
+     * The GitHub IDs of the people among a repository's top contributors, remembered for a day so that everyone whose
+     * packages come from it shares one request. A repository GitHub no longer has, one without history, or one whose
+     * history GitHub finds too large to list has none. When GitHub throttles it, the lookup fails so it is tried again
+     * later, instead of quietly finding nobody.
+     *
+     * @return array<int, string>
+     */
+    private function contributors(string $repository): array
+    {
+        try {
+            return Cache::remember("dependency-contributors:{$repository}", 86400, function () use ($repository): array {
+                $response = $this->github->request()->get("repos/{$repository}/contributors", ['per_page' => self::Contributors]);
+
+                if ($response->notFound() || $response->noContent() || $response->fluent()->string('message')->contains('too large')) {
+                    return [];
+                }
+
+                return $response->throw()
+                    ->collect()
+                    ->where('type', 'User')
+                    ->mapInto(Fluent::class)
+                    ->map(fn (Fluent $contributor): string => $contributor->string('id')->value())
+                    ->values()
+                    ->all();
+            });
+        } catch (HttpClientException $httpClientException) {
+            throw_if($httpClientException instanceof RequestException && in_array($httpClientException->response->status(), [403, 429], true), $httpClientException);
+
+            return [];
+        }
+    }
+
+    /**
+     * A registry's document about a package, empty when the registry doesn't know it.
+     *
+     * @return array<array-key, mixed>
+     */
+    private function registry(string $url): array
+    {
+        $response = $this->http()->get($url);
+
+        return $response->notFound() ? [] : (array) $response->throw()->json();
+    }
+
+    private function http(): PendingRequest
+    {
+        return Http::withUserAgent(sprintf('devnull/1.0 (+%s)', config()->string('app.url')))->connectTimeout(3)->timeout(5);
+    }
+}