🚀 SCALE - DevOps-as-a-Service
SCALE is NetApp's solution for providing DevOps-as-a-Service to application teams, combining a well-defined software development framework and workflow with the underlying infrastructure for the deployment and execution of applications - from development and test all the way through to production.
SCALE implements container technology and is purpose-built for delivering and running application Microservices and web-based UI components, providing:
- ✅ A continuous pipeline to build and deploy components
- ✅ Cloud-based environments in which components run
- ✅ Role-based access control across all environments
🧱 Technology Stack
SCALE is built on top of these core technologies:
| Technology | Description |
|---|---|
| Docker | Container technology - a very small execution environment for microservices and small web UI components |
| Kubernetes | Orchestration environment for Docker containers; organizes containers into deployable units for high availability and scalability |
| Rancher | A Kubernetes distribution from SUSE with additional management, security, resource isolation, and log collection capabilities |
| Git | Source code repository within Azure DevOps, hosting application code according to SCALE conventions |
| Artifactory | Repository for container images produced by the build process |
| Helm Charts | Kubernetes-agnostic mechanism for defining deployable application units using a templating mechanism |
| Azure DevOps | Technology for CI (Build) and CD (Deploy) pipeline stages |
To make changes to source code in Git, developers create a feature branch from the master branch, make their updates locally, and push the branch to the remote repository. A pull request (PR) is then opened to propose merging those changes back into master. Team members review the PR, leave comments, and request changes if needed. When a PR is created, it automatically triggers a Release pipeline that builds the code and deploys it to the dev environment (Workspace). The pipeline then waits for human approval before promoting the deployment to Stage and Production. Once the changes are successfully deployed to Production, the PR can be completed and merged into master.
📦 Provisioned Components
When onboarding or modifying an application in SCALE, there are 3 types of spaces that can be provisioned:
🔧 Workspaces
The space where developers produce application code. Includes source code repositories, build/deployment pipelines, an image repository, and other development tools.
Workspaces are hosted in Kubernetes on AWS with limited memory/CPU quotas and a 2-hour time span for deployed components. They are designed for development and initial unit testing only.
🏠 Hostspaces
Kubernetes environments for deploying application components after development and unit testing are complete. Components in Hostspaces are expected to remain deployed indefinitely.
- Used for Staging, Pre-Production, and Production environments
- Hosted in-house in NetApp data centers
- Quotas can be selected to accommodate varying memory, storage, and CPU needs
💾 Dataspaces
Predefined database stacks within SCALE that can vary in size and specifications.
See our DB Catalog for available database types and configurations.
📋 Provisioning Procedures
Onboarding an Application
The onboarding process is handled through a ServiceNow portal named "Onboard CloudOne Application". Applications must first be registered in ServiceNow before onboarding can begin.
- Inventory update - ServiceNow CMDB is updated with application and component information
- Workspace provisioned - fixed-size quota (short-term usage only)
- App Stacks provisioned - one per application component (microservices, front-end UIs)
- Hostspaces provisioned - based on selected size, with network connectivity specified (external or internal)
- AD groups generated - propagated to Rancher (requires ~30 min pause)
- Azure DevOps - pipelines and projects created
- Artifactory - image repository area provisioned
- Kubernetes namespaces - provisioned in AWS (Workspace) and NetApp datacenters (Hostspaces)
- Git repositories - generated for each App Stack with initial file structure
- Helm Charts - generated for deployment (can be replaced with Kustomize files)
Managing an Application
Managing an already-onboarded application is similar to onboarding but skips one-time setup steps. Through the ServiceNow portal, you can:
- Add new App Stacks (components)
- Provision additional Hostspaces
- Provision Dataspaces
Dataspaces cannot be provisioned during initial onboarding - they must be provisioned from the "Manage CloudOne DevOps Application" portal.
🔐 Role-Based Access Control (RBAC)
Access and permissions within SCALE are determined by Active Directory group assignments. For every application onboarded, the following groups are generated (where XXX is the application code):
NG Group Descriptions
| AD Group | Role | Description |
|---|---|---|
ng-ngdc-XXX-admin | Admin | Application owners ONLY. Used ONLY for granting access to other NG groups. |
ng-ngdc-XXX-dev | Developer | All developers with access to git, pipelines, etc. |
ng-ngdc-XXX-leads | Dev Leads | Authorized to approve deployments to Stage Hostspaces |
ng-ngdc-XXX-ops | Operations | Operational support; can approve production deployments |
ng-ngdc-XXX-dba | DBA | Same permissions as dev group |
ng-ngdc-XXX-biz | Business | Approves testing completion and readiness for Stage |
Admin group members can assign users to any group via the NEAT application: https://neat.netapp.com
Azure DevOps Repository Permissions
| Permission | Groups |
|---|---|
| Read-Write | ng-ngdc-XXX-dev, ng-ngdc-XXX-leads, ng-ngdc-XXX-dba |
| Read-Only | ng-ngdc-XXX-ops, ng-ngdc-XXX-biz |
CyberArk Safe Permissions
| Landscape | Safe | Groups |
|---|---|---|
| Dev | S-D-C-RNCHR-CNJR-xxx | ng-ngdc-XXX-dev, ng-ngdc-XXX-leads, ng-ngdc-XXX-dba |
| Stage | S-S-C-RNCHR-CNJR-xxx | ng-ngdc-XXX-ops, ng-ngdc-XXX-leads, ng-ngdc-XXX-dba |
| Prod | S-P-C-RNCHR-CNJR-xxx | ng-ngdc-XXX-dev, ng-ngdc-XXX-dba |
Rancher/Kubernetes Permissions
Each user should only request and be granted access to a single Access Package below. Requesting access to multiple access packages for a single application/appcode can cause issues and will be periodically audited.
For Kubernetes access, users will need to log in using their a-sso account. Users can request an a-sso account using the A-SSO Admin Account Request form. Users will need to request access to the necessary "Access Package" by logging into the "My Access" portal using their a-sso and following the steps in KB0007629. Once approved, they can log into Kubernetes using their a-sso. Keep in mind that it might take 1-2 hours for the access to apply once approved.
The individuals who approve the following access packages are the Application Development Owner and Application Support Owner. Please note that if these application owners change, the Access Package approvers are not updated with the new owners.
WS = Workspace, HS = Hostspace, DS = Dataspace (reference)
| Access Package | CG Group | WS + HS (dnt) | HS (stg) | HS (prd) | DS (all) |
|---|---|---|---|---|---|
| IT DevOps <appcode> Developers | cg-pam-devops-<appcode>-dev | Edit | View | View | View |
| IT DevOps <appcode> Leads | cg-pam-devops-<appcode>-leads | Edit | Edit | View | View |
| IT DevOps <appcode> Operations | cg-pam-devops-<appcode>-ops | Edit | Edit | Edit | Edit |
| IT DevOps <appcode> DBA | cg-pam-devops-<appcode>-dba | View | View | View | Edit |
⚙️ SCALE CI/CD Pipeline
Continuous Integration - Build & Quality Checks
The CI pipeline stage is triggered by a merge request to the master branch in Git. Upon successful completion, a container image is generated and readied for deployment.
Branch Policy Prerequisites
Before the CI pipeline begins, Azure DevOps enforces these Branch Policies:
- ✅ All comments on the branch must be resolved
- ✅ All work must be linked within a hierarchy
- ✅ PR Check criteria must pass for merge completion
- ✅ Multiple merges are consolidated before pipeline initiation
CI Pipeline Workflow
| Step | Action |
|---|---|
| 1 | Merge request triggers the flow |
| 2 | Validate pipeline variables (appCode, appVersion as valid SemVer) |
| 3 | Check version is not a duplicate of a previous release |
| 4 | Build the code (Docker S2I or explicit build commands) |
| 5 | Generate Helm Chart and publish to Artifactory |
| 6 | Run automated unit tests |
| 7 | Generate Docker image and publish to Artifactory |
| 8 | Deploy to Workspace (OpenShift in AWS) |
| 9 | Run code scan (e.g. Jacoco) |
| 10 | Run security scan (JFrog XRay) |
| 11 | Decorate scan results to the Git pull request |
| 12 | Verify security scan score meets threshold |
| 13 | Complete Pull Request - merge new code to master |
Exemptions are granted by the security team. The "Security Scan Exempted" flag can override the threshold. An exemption link will be decorated to the PR.
Continuous Delivery - Deployment Pipeline
The CD pipeline is triggered by successful CI completion and deploys code through all environments, each pending manual approval.
| Step | Stage | Approver |
|---|---|---|
| 1 | Deploy to Workspace via Helm/Kustomize | - |
| 2 | Pause for Unit Test completion | Dev Leads Group |
| 3 | Finalize release tags and version numbers | - |
| 4 | Deploy to Staging Hostspace(s) | - |
| 5 | Pause for UAT testing approval | Business Approvers Group |
| 6 | Update PR Check status, complete merge | - |
| 7 | Generate -PRD tagged artifact (immutable) | - |
| 8 | Deploy to Pre-Production Hostspace(s) | - |
| 9 | Pause for Production approval | Operations Group |
| 10 | Deploy to Production Hostspace(s) | - |
All deployments can be approved for immediate execution or scheduled for a future date and time.
References
- Semantic Versioning - https://semver.org/