Service Rollouts

In VESSL, you can seamlessly orchestrate the entire deployment process, from performing inference tasks on servers to directing traffic on the actual servers, all in one place. The Endpoint section in the Monitor tab allows you to specify how traffic should be sent to the servers. This eliminates the need for manual intervention in every deployment, reducing the risk of errors and the hassle of repetitive tasks.

VESSL introduces a powerful Rollout feature, enabling you to define the deployment process step by step in advance and execute it automatically when needed. Additionally, it offers user-friendly features like email notifications and deployment monitoring, ensuring that you can implement uninterrupted deployments with ease.

Let's explore how VESSL's Rollout can be used to implement various deployment patterns:

• Rolling Update: With VESSL's Rollout, you can smoothly transition to a new revision by sequentially directing traffic to it. This ensures a gradual and uninterrupted rollout of the new revision.

• Blue-Green Deployment: In situations where two revisions of an inference server should never run simultaneously, VESSL simplifies the process. It allows you to deploy the new revision with the same workload as the existing one and then seamlessly switch endpoints, ensuring a continuous rollout of the next version.

• Canary / A-B Testing: By directing a portion of the traffic to the new revision, you can gather metrics and make informed decisions for the next deployment steps.

Rollout lists

You can view and modify ongoing rollouts in the List page.

Rollout Lists

• Pause: Temporarily halt an ongoing rollout. The current step continues before pausing until manually resumed.

• Resume: Restart a paused rollout.

• Terminate: Completely stop an ongoing rollout, marking it as a failure.

• Reproduce: Create a new rollout identical to an existing one.

Create a Rollout

To create a new rollout, click the "New rollout" button on the Rollout tab.

New Rollout

On the rollout creation page, you can combine messages and "Steps" to define the order of deployment. Here's what each step entails:

Rollout Step Create

• CreateNewRevision: Generate a new revision, using the same UI as Service Revision
• UpdateRevisions: Modify the deployment status and auto-scaling settings of existing or newly created revisions.

  • Revision Number: Select the target revision.

  • Min/Max Replica: Set the minimum and maximum auto-scaling range.

  • Running: Decide whether to deploy or halt the selected revision, useful for scenarios involving the deployment of a new revision alongside the shutdown of an existing one.

• ChangeEndpointConfig: Adjust traffic routing settings for the endpoint, following the UI described in Change Endpoint Configuration
• Wait: Temporarily pause the rollout based on predefined settings. Three options are available:

  • Time: Automatically resume after a set time.

  • Manual: Wait until a user manually resumes the rollout from the dashboard.

  • Metric (planned feature): Automatically resume when specific metrics criteria are met.

• SendNotification: Dispatch notifications based on predefined settings, typically right before deployment completion or when user confirmation is required.

  • Channel:

    • Slack: Send notifications to a Slack endpoint configured in the organization settings.

    • Email: Dispatch notifications to a designated email address.

  • Fail Action: Determine the action when a notification fails.

    • Skip: Continue with remaining steps.

    • Abort: Treat the rollout as a failure.

Once steps are defined, you can review, modify, reorder, or delete them in the Step detail list.

• Drag the leftmost handle to change the step order.

• Use the Edit option in the Action column to review and modify step definitions.

• Unwanted steps can be deleted using the Delete icon in the Action column.

Reproduce Rollouts

You can clone existing rollouts by clicking the "Reproduce" button associated with a specific rollout. Typically, once a deployment pattern is defined, it remains largely unchanged, with only revision information varying. Leveraging the "Reproduce" feature allows for the straightforward repetition of previously defined rollout steps. This offers an efficient way to maintain consistency in your deployment processes.

Rollout Reproduce