Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1443,6 +1443,27 @@ To install an upgrade package:
ghe-upgrade UPGRADE-PACKAGE-FILENAME
```


{% ifversion ghes > 3.20 %}

Beginning with upgrades in version 3.21 operators may run many of the upgrade operations without requiring a maintenance window using phased execution.

First run operations which do not require a maintenance window by triggering the pre-upgrade phase

```shell
ghe-upgrade --phase pre-upgrade UPGRADE-PACKAGE-FILENAME
```

Once that is complete operators may complete the upgrade by running the final steps after a maintenance window has been scheduled

```shell
ghe-upgrade --phase upgrade UPGRADE-PACKAGE-FILENAME
```

The upgraded {% data variables.product.prodname_enterprise %} host will be rebooted by this operation.

{% endif %}

{% data reusables.enterprise_installation.command-line-utilities-ghe-upgrade-rollback %}

### ghe-upgrade-scheduler
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,305 @@
---
title: Enterprise Server Upgrade Automation
intro: You can use the ghes-manage API or `es` plugin to automate upgrade operations in Enterprise Server environments.
versions:
ghes: '>= 3.21'
shortTitle: Upgrade Enterprise Server
contentType: how-tos
---

You can upgrade a GitHub Enterprise Server instance using the Manage {% data variables.product.prodname_ghe_server %} API or the `gh es` CLI extension. These tools automate the process of downloading the upgrade package, running pre-upgrade checks, and applying the new version.

## Prerequisites

- Back up your data with [GitHub Enterprise Server Backup Utilities](https://github.com/github/backup-utils#readme).
- Schedule a maintenance window for end users.
- Ensure you have authentication credentials for the Manage {% data variables.product.prodname_ghe_server %} API. For more information, see [AUTOTITLE](/rest/enterprise-admin#authentication).

## Step 1: Download the upgrade package

Download the upgrade package to the instance.

### Using the CLI

```shell
# Download a specific version
gh es upgrade download --version VERSION

# Or download the latest available version
gh es upgrade download
```

### Using the API

```shell
curl -L \
-X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/download \
-d '{"version":"VERSION"}'
```

## Step 2: Monitor download progress

Confirm the download has completed before proceeding.

### Using the CLI

```shell
gh es upgrade download status
```

### Using the API

```shell
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/download/status
```

Wait until `status` shows `COMPLETED`.

## Step 3: Apply the upgrade's pre-upgrade phase

Apply the upgrade. This runs both the pre-upgrade and upgrade phases sequentially.

### Using the CLI

```shell
gh es upgrade apply --version VERSION --phase pre-upgrade
```

### Using the API

```shell
curl -L \
-X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/apply \
-d '{"version":"VERSION", "phase":"pre-upgrade}'
```


## Step 4: Monitor pre-upgrade progress

Monitor the upgrade until it completes. The instance will reboot once the upgrade finishes.

### Using the CLI

```shell
gh es upgrade status --verbose
```

### Using the API

```shell
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
"https://HOSTNAME:8443/manage/v1/upgrade/status?verbose=true"
```

Wait until `status` shows `completed` and `is_running` shows `false`.

## Step 5: Enable maintenance mode

Enable maintenance mode.

### Using the CLI

```shell
gh es maintenance set --enabled true
```

### Using the API

```shell
curl -L \
-X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/maintenance \
-d '{"enabled":true}'
```

## Step 6: Apply the upgrade's upgrade phase

Apply the upgrade. This runs both the pre-upgrade and upgrade phases sequentially.

### Using the CLI

```shell
gh es upgrade apply --version VERSION --phase pre-upgrade
```

### Using the API

```shell
curl -L \
-X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/apply \
-d '{"version":"VERSION", "phase":"pre-upgrade}'
```

## Step 7: Verify and disable maintenance mode

After the upgrade completes:

First, confirm the release version has been updated.

### Using the CLI

```shell
gh es release version
```

### Using the API

```shell
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/version
```

Then, disable maintenance mode.

### Using the CLI

```shell
gh es maintenance set --enabled false
```

### Using the API

```shell
curl -L \
-X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/maintenance \
-d '{"enabled":false}'
```

## Upgrading a high availability deployment

For instances with a high availability (HA) replica, the download and pre-upgrade phases are non-disruptive and can run across all nodes at once. UUID targeting is only needed for the upgrade phase itself, which triggers the reboot. This lets you control the order nodes reboot in: upgrade the replica first, then the primary.

To retrieve node UUIDs, run `gh es config get-metadata` or query `GET /manage/v1/config/nodes`.

### Using the CLI

```shell
# Download the package to all nodes
gh es upgrade download --version VERSION

# Wait for download to complete on all nodes
gh es upgrade download status

# Run pre-upgrade on all nodes at once (non-disruptive)
gh es upgrade apply --version VERSION --phase pre-upgrade

# Wait for pre-upgrade to complete
gh es upgrade status --verbose

# Enable maintenance mode
gh es maintenance set --enabled true

# Upgrade the replica first (triggers reboot)
gh es upgrade apply --version VERSION --phase upgrade --uuid REPLICA-UUID
gh es upgrade status --uuid REPLICA-UUID --verbose

# After the replica finishes, upgrade the primary
gh es upgrade apply --version VERSION --phase upgrade --uuid PRIMARY-UUID
gh es upgrade status --uuid PRIMARY-UUID --verbose

# Verify replication health and version, then disable maintenance mode
gh es replication status
gh es release version
gh es maintenance set --enabled false
```

### Using the API

```shell
# Download the package to all nodes
curl -L -X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/download \
-d '{"version":"VERSION"}'

# Wait for download to complete on all nodes
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/download/status

# Run pre-upgrade on all nodes at once (non-disruptive)
curl -L -X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/apply \
-d '{"version":"VERSION","phase":"pre-upgrade"}'

# Wait for pre-upgrade to complete
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
"https://HOSTNAME:8443/manage/v1/upgrade/status?verbose=true"

# Enable maintenance mode
curl -L -X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/maintenance \
-d '{"enabled":true}'

# Upgrade the replica first (triggers reboot)
curl -L -X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/apply \
-d '{"version":"VERSION","phase":"upgrade","uuid":"REPLICA-UUID"}'

# Monitor replica upgrade
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
"https://HOSTNAME:8443/manage/v1/upgrade/status?uuid=REPLICA-UUID&verbose=true"

# After the replica finishes, upgrade the primary
curl -L -X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/upgrade/apply \
-d '{"version":"VERSION","phase":"upgrade","uuid":"PRIMARY-UUID"}'

# Monitor primary upgrade
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
"https://HOSTNAME:8443/manage/v1/upgrade/status?uuid=PRIMARY-UUID&verbose=true"

# Verify replication health and version, then disable maintenance mode
curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/replication/status

curl -L \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/version

curl -L -X POST \
-u "api_key:ROOT-SITE-ADMINISTRATOR-PASSWORD" \
-H "Content-Type: application/json" \
https://HOSTNAME:8443/manage/v1/maintenance \
-d '{"enabled":false}'
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: Upgrade Automation With API or CLI
intro: 'You can automate appliance upgrade operation using the ghes-manage API and `es` plugin.'
versions:
ghes: '>= 3.21'
children:
- /enterprise-server-upgrade-automation
---
1 change: 1 addition & 0 deletions content/admin/upgrading-your-instance/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@
- /preparing-to-upgrade
- /performing-an-upgrade
- /troubleshooting-upgrades
- /automation-via-cli-api
shortTitle: Upgrade your instance
---
Original file line number Diff line number Diff line change
Expand Up @@ -95,3 +95,26 @@ To upgrade a multi-node {% data variables.product.prodname_ghe_server %} environ
{% data reusables.enterprise_installation.replication-status %} {% data reusables.enterprise_installation.replication-status-upgrade %}
{% data reusables.enterprise_installation.multiple-node-repeat-upgrade-process %}
{% data reusables.enterprise_installation.disable-maintenance-mode-after-replica-upgrade %}

## Upgrading an instance using phased upgrade execution

Phased upgrade execution allows {% data variables.product.prodname_ghe_server %} operators running versions 3.22 or greater better control over downtime-inducing actions by isolating those actions to their own phase. To use phased execution perform the following after downloading the upgrade package:
1. Run the package's pre-upgrade phase

```shell
ghe-upgrade --phase pre-upgrade GITHUB-UPGRADE.pkg
```

1. Enable maintenance mode and wait for all active processes to complete on the {% data variables.product.prodname_ghe_server %} instance. See [AUTOTITLE](/admin/configuration/configuring-your-enterprise/enabling-and-scheduling-maintenance-mode).

> [!NOTE] When upgrading the primary node in a high availability configuration, the instance should already be in maintenance mode if you are following the instructions in [Upgrading the primary node with an upgrade package](#upgrading-the-primary-node-with-an-upgrade-package).
1. Run the upgrade phase

```shell
ghe-upgrade --phase pre-upgrade GITHUB-UPGRADE.pkg
```

1. Optionally, after the upgrade, validate the upgrade by configuring an IP exception list to allow access to a specified list of IP addresses. See [AUTOTITLE](/admin/configuration/configuring-your-enterprise/enabling-and-scheduling-maintenance-mode#validating-changes-in-maintenance-mode-using-the-ip-exception-list).
1. For single node upgrades, perform any post-upgrade tasks including disabling maintenance mode so users can use {% data variables.location.product_location %}.

> [!NOTE] After you upgrade an instance in a high availability configuration, you should remain in maintenance mode until you have upgraded all of the replica nodes and replication is current. See [Upgrading additional nodes with an upgrade package](#upgrading-additional-nodes-with-an-upgrade-package).
7 changes: 6 additions & 1 deletion content/billing/concepts/budgets-and-alerts.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,12 @@ For metered products such as {% data variables.product.prodname_actions %}, {% d
Each budget has a type and a scope that define which paid use contributes to spending against the budget.

* **Type**: Defines which metered product or SKU is measured.
* **Scope**: Defines whether the budget applies to the whole account, or to a subset of repositories, organizations, cost centers (enterprise only), or users. User-scoped budgets are currently only supported for {% data variables.product.prodname_copilot_short %} {% data variables.product.prodname_ai_credits_short %}. There are two types: a universal budget that applies to all licensed users by default, and individual budgets that override the universal for specific users. See [AUTOTITLE](/copilot/concepts/billing/budgets-for-usage-based-billing).
* **Scope**: Defines whether the budget applies to the whole account, or to a subset of repositories, organizations, cost centers (enterprise only), or users. User-scoped budgets are currently only supported for {% data variables.product.prodname_copilot_short %} {% data variables.product.prodname_ai_credits_short %}, and have three scopes:
* **Universal**: applies to all licensed users by default
* **Cost center user-level**: applies to every user in a cost center
* **Individual**: overrides the above for specific users

For more information about how user-level budgets work and how they interact with other budget controls, see [AUTOTITLE](/copilot/concepts/billing/budgets-for-usage-based-billing).

## Budget alerts

Expand Down
Loading
Loading