Matrix42 Enterprise hosted in Containers: Early Adopters Program

Explore how hosting enterprise service management in containers empowers early adopters to enhance efficiency and drive innovation.

About This Program

Disclaimer

The Early Adopters Program is intended to provide early access to dedicated deliverables of Enterprise ESM Platform that can be hosted in Containerized infrastructure. The primary purpose is to establish a short feedback cycle. This allows the Matrix42 to collect feedback and address issues as early as possible.

The solution is based on a dedicated branch that is planned to be made generally available with version 27.1. Therefore, using the deliverable from Early Adopters Program in a production environment is not permitted, unless agreed with Product Management.

All feedback, questions, and requests related to Early Adopters Program are handled directly by the Matrix42 Engineering. Please do not contact Matrix42 Support for any issues related to it.

Contact: itsm.feedback@matrix42.com, vadim.zinoviev@matrix42.com

The Early Adopters Program may include features that are still under active development and are planned to be made generally available in either version 26.2 or version 27.1.

Issues related to features that are not specifically related to ESM Platform hosted in Containers functionality can be reported only after feature is made generally available. 

 

Until that, the solution provides a limited scope of functionality. The available feature set will be expanded incrementally through regular updates.

Only Sandboxed Extensions are supported in the ESM Platform hosted in Containers.

Purpose

  • Deliver the experience of hosting Matrix42 Enterprise in containers in Private Cloud or on-prem.
  • Collect feedback and polish the process before general availability.
  • Provide an environment for developing and testing sandboxed extensions running on Linux.

Delivery Model

The solution is delivered as a set of Docker images distributed over the Harbor Container Registry: https://registry.m42cloud.com/

To get access to the Container Registry, request a Harbor account from the Matrix42 Cloud Team by email: cloud_ops@matrix42.com

curl -u '{user}:{password}' https://registry.m42cloud.com/api/v2.0/projects/esmp/repositories

New image versions are uploaded to the registry regularly, 1–2 times per month, and an email with the list of delivered features is sent to Program participants.

Roadmap: Missing Features and Upcoming Deliveries

Feature Status Milestone Description
Reliable Tenant Update In Progress 1 The first version handles only the successful update path and does not yet handle update failures correctly. Aims to guarantee reliable ESM tenant updates with rollback, without significant impact on uptime (near-zero downtime).
Tenant Monitoring In Progress 1 Activates Prometheus for monitoring event collection and delivers default Grafana dashboards.
Linux Web Services Container To do 2 Web Services currently run in a Windows container based on the .NET Framework process on IIS. After this change, Web Services will run in a Linux container based on .NET 10.
Enforce Sandboxed Extensions To do 3 The product will work only with sandboxed extensions: only sandboxed extensions can be installed, and the Configuration Project will generate only sandboxed extensions.
Web Installer on Functional Worker Engine To do 3 Discontinues the "WmInstaller" web application as a standalone solution; its logic moves partially to M42Services and partially to the Functional Worker.
Unattended Extension Updates To do 4 Introduces a single button in the UUX console and in the Tenant Management console to automatically update all installed Extension Gallery extensions to their latest version.
Automatic Deployment of Certificates in Private Cloud To do 4 Automatically delivers the relevant certificates to Private Cloud directly from the Matrix42 License Server.
File Storage Provider for S3-Compatible Storage To do TBD Introduces the ability to store ESM files in S3-compatible storage hosted in the k8s cluster.

Concept

What Is Matrix42 Tenant Management?

Matrix42 Tenant Management is a web application that Matrix42 delivers as a Helm chart. You install it once on your Kubernetes cluster, and it manages the full lifecycle of ESM Platform tenants: installing new tenants, updating them to new versions, and decommissioning them when they are no longer needed,etc.

It is the management plane, not the workload. Once a tenant is up and running, ESM Platform serves your end users entirely on its own — Matrix42 Tenant Management steps in only when you trigger a lifecycle operation. One installation handles all tenants on the same cluster. Each tenant lives in its own isolated Kubernetes namespace, so tenants cannot interfere with each other.

What Is an Tenant?

Each Enterprise ESM Platform tenant is a self-contained deployment of the application running in its own Kubernetes namespace. It consists of three Kubernetes workloads and a set of SQL Server databases:

  • An IIS-based web application running in a Windows pod
  • A Linux-based controller proxy for third-party integrations
  • A Linux-based background job dispatcher

The SQL Server databases are hosted on your existing SQL Server — Matrix42 Tenant Management does not deploy or manage a database server itself. The install pipeline creates the databases and their schema.

What Is Argo Workflows?

Argo Workflows is a Kubernetes-native pipeline engine bundled with Matrix42 Tenant Management. You do not install or manage it separately — it is included as part of the Helm chart.

When you trigger an operation (for example, installing a new tenant), Matrix42 Tenant Management does not execute the steps itself. Instead, it builds a workflow description and hands it off to Argo, which runs each step as a short-lived pod on your Linux nodes. The workflow is stored as a Kubernetes resource, which means:

  • It survives restarts. An in-flight installation keeps running even if you restart or upgrade the application.
  • It retries automatically. If a step fails due to a transient error (a SQL timeout, a network blip), Argo retries it with exponential back-off before reporting failure.
  • You can resume from failure. If a run fails permanently, you can retry it from the failed step — everything that succeeded before is not repeated.
  • Steps run in parallel where possible. Independent steps run concurrently to keep the total operation time as short as possible.
  • Every step is auditable. Start time, end time, result, and full log output are recorded for every step. Logs are archived in associated S3-compati.

How Installing a Tenant Works

When you submit a new tenant installation, Matrix42 Tenant Management composes a pipeline and hands it to Argo. The pipeline creates the Kubernetes namespace and all the RBAC within it, copies in your TLS certificate and ESM certificates, generates all cryptographic keys and credentials entirely inside the cluster (nothing leaves the cluster), creates the SQL databases and installs the schema, and then deploys each ESM workload in the right order — waiting for the web application to fully initialize before starting the dispatcher.

The whole process takes around 20–35 minutes. Most of that time is IIS startup in the Windows container. At the end, you receive the tenant's first administrator credentials — the account used to log in to the new Enterprise ESM Platform deployment.

If any step fails, you can see exactly which step failed and read its log output. From there, you can resume the pipeline from the failed step without restarting from scratch.

How Updating a Tenant Works

The update pipeline upgrades a running tenant to a new Enterprise ESM Platform version. It is designed around a single critical constraint: the main database migration is not reversible, so the pipeline enforces a mandatory human approval step before any downtime begins.

The pipeline first validates that the new version is compatible with the current database, then pauses and waits for an operator to confirm before proceeding. Only after that approval does it scale down the web application and dispatcher, run the main migration, switch all three workloads to the new image version, and bring everything back up. The downtime window is typically a few minutes.

Authentication and Access Levels

Logging In With ServiceAccount Tokens

Matrix42 Tenant Management authenticates users via Kubernetes ServiceAccount tokens. Each user corresponds to a Kubernetes ServiceAccount in the tmc-system namespace. To log in, a user presents a time-bounded token generated with kubectl:

kubectl create token <username> -n tmc-system --duration=8760h

That token is entered on the login page, or passed as a bearer token in API calls. The token is validated against the Kubernetes API — there are no passwords stored in the application.

The first user is an admin account named tmc-user-admin that is created automatically on startup. The cluster administrator generates its token and hands it to the first person to log in.

Access Levels

There are three access levels:

  • Admins configure the cluster (S3, Loki) and manage users.
  • Operators install, update, and decommission tenants, and approve update pipeline gates.
  • Viewers have read-only access to tenant status and run history.

Where Data Lives

Matrix42 Tenant Management stores no application state in a relational database. Everything lives in Kubernetes itself, so restarting or upgrading the application pod loses nothing.

  • Cluster configuration (your S3 and Loki connection details) is held in Kubernetes Secrets in the tmc-system namespace, managed through the settings screen.
  • Pipeline run state is stored in Argo Workflow custom resources — Argo is the authoritative source; the application reads from it in real time.
  • Tenant secrets (database connection strings, cryptographic keys, certificates) are stored in Kubernetes Secrets inside each tenant's namespace and never touch the application's memory or disk.
  • Pipeline step logs are archived to S3-compatible storage.
  • User accounts are Kubernetes ServiceAccounts.

Requirements and Prerequisites

Before installing Matrix42 Tenant Management, ensure that the Kubernetes cluster and all external dependencies meet the following requirements.

Kubernetes Cluster

The solution requires a conformant Kubernetes cluster with version 1.26 or later.

The solution is designed to be Kubernetes-distribution independent and can be deployed on platforms such as:

  • AKS
  • EKS
  • GKE
  • RKE2
  • k3s
  • Bare-metal Kubernetes installations

The exact support matrix may depend on the specific Private Cloud ESM release.

Node Types

Your cluster needs two kinds of worker nodes running side by side:

Linux Nodes

Linux nodes host:

  • The Tenant Management application.
  • Argo Workflows.
  • Workflow executor pods.
  • Linux-based ESM workloads, like “Setup”, “Controller”, “Functional Worker”, later “Web Services”

Ubuntu 22.04 LTS is a recommended starting point. Other supported Linux distributions may also be used.

As an initial sizing guideline, plan for at least: 4 vCPUs, 8 GB RAM per Linux worker node.

Actual capacity requirements depend on the number of tenants and concurrent lifecycle operations.

Windows Nodes

Windows nodes (Windows Server 2022 with containerd) are required because temporarily the ESM web application is an IIS-based Windows container and for Functional Worker to run tasks required Windows capability. Linux nodes alone cannot host a running ESM tenant. Plan for at least 4 vCPU and 16 GB RAM per Windows node.

Ingress and TLS

An Ingress controller must be installed and operational before deploying Tenant Management.

Supported Ingress implementations may include:

  • NGINX Ingress.
  • Traefik
  • Contour
  • HAProxy
  • Other Kubernetes-compatible Ingress implementations.

All addresses share a single wildcard domain. For example, if your domain is esmp.company.com, the console lands at https://tmc-ui.esmp.company.com and a tenant named acme lands at https://acme.esmp.company.com. You need a wildcard TLS certificate for *.esmp.company.com stored as a Kubernetes secret somewhere in the cluster — Helm copies it to the right place during installation.

Network Access

The Kubernetes cluster must be able to communicate with the following external services:

Destination Protocol/Port Purpose
Matrix42 Container Registry
registry.m42cloud.com
HTTPS/443 Pulling Matrix42 container images
SQL Server TCP/1433 or configured port Creating and managing ESM databases
S3-compatible storage HTTPS or configured endpoint Storing workflow logs and ESM-related artifacts
Loki HTTPS or configured endpoint Collecting application logs
Prometheus    

The cluster must also have a default Kubernetes StorageClass.

Argo Workflows requires persistent storage for some of its internal operations. The default StorageClass is used when no explicit storage class is configured.

S3-Compatible Object Storage

Deployment of Enterprise ESM platform to Containers requires access to S3-compatible object storage. Object storage is used for:

  • Storing extension artifacts in shared buckets.
  • Providing dedicated storage for individual tenants.
  • Storing tenant files and attachments (planned).
  • Storing other platform file-based data, such as Import Definitions (GDI) sources.
  • Archiving Argo workflow logs.

The storage can be hosted inside or outside the Kubernetes cluster.

Examples include:

  • Amazon S3.
  • MinIO
  • Other S3-compatible storage solutions.

Each tenant requires a dedicated bucket or equivalent isolated storage location with appropriate access permissions.

SQL Server

A supported SQL Server instance must be reachable from the Kubernetes cluster.

The SQL Server account used during tenant installation must have sufficient permissions to:

  • Create the required databases.
  • Create and modify database objects.
  • Apply the database schema and migrations.

The exact SQL Server version and permission requirements may vary by Enterprise ESM Platform release and must be validated against the applicable release requirements.

Installing Tenant Management

The Helm chart is in final preparation. The commands below reflect the planned interface; exact parameter names will be confirmed in values.yaml at release.

 

Before You Install

You need the following in place before running helm install.

Required from Matrix42

  • Harbor Container Registry credentials or a robot account with permission to pull the required images.
  • Access to the Matrix42 container registry.
  • ESM certificates ZIP files required for tenant installation.

Required in Your Infrastructure

  • A Kubernetes cluster with Linux and Windows worker nodes.
  • A functioning Kubernetes Ingress controller.
  • A wildcard TLS certificate for the configured domain.
  • A SQL Server 2019 or newer, reachable from cluster nodes on TCP port 1433. The SQL account you provide must have dbcreator rights.
  • A default Kubernetes StorageClass.
  • Network connectivity to the Matrix42 Container Registry.
  • S3-compatible object storage.
  • A Loki endpoint if centralized application logging is required.

Running the Helm Install

Step 1 — Log in to the registry

 helm registry login registry.m42cloud.com \
   --username <robot-account> \
   --password <token>

 This is required once per machine (or per CI/CD session) before any helm install or helm upgrade.

 Step 2 — Install (first time)

Latest version:

helm install tenant-management oci://registry.m42cloud.com/esmp/tenant_management_helm \
   --namespace tmc-system --create-namespace \
   --set harbor.registry=registry.m42cloud.com \
   --set harbor.username=<robot-account> \
   --set harbor.password=<token> \
   --set tmc.hostname=tmc-ui \
   --set tmc.domain=esmp.your-domain.com \
   --set tls.sourceNamespace=<namespace-where-cert-lives> \
   --set tls.sourceSecretName=<wildcard-tls-secret-name>

Or specific version:

 helm install tenant-management oci://registry.m42cloud.com/esmp/tenant_management_helm \
   --version 1.0.9 \
   --namespace tmc-system --create-namespace \
   --set harbor.registry=registry.m42cloud.com \
   --set harbor.username=<robot-account> \
   --set harbor.password=<token> \
   --set tmc.hostname=tmc-ui \
   --set tmc.domain=esmp.your-domain.com \
   --set tls.sourceNamespace=<namespace-where-cert-lives> \
   --set tls.sourceSecretName=<wildcard-tls-secret-name>

Parameter notes:

  • harbor.registry — Registry hostname provided by Matrix42 (e.g. registry.m42cloud.com)
  • harbor.username — Harbor robot account username supplied by Matrix42
  • harbor.password — Harbor robot account token supplied by Matrix42
  • tmc.hostname — Subdomain for the console (e.g. tmc-ui)
  • tmc.domain — Shared base domain for the console and all tenants (e.g. esmp.company.com)
  • tls.sourceNamespace — Namespace where your wildcard TLS secret currently lives
  • tls.sourceSecretName — Name of the wildcard TLS secret

 

--namespace tmc-system is the only valid value. The namespace tmc-system is hardcoded throughout the chart and baked into the application image — every resource is always created there regardless of what you pass. The flag is still required so that Helm stores its release metadata in tmc-system, which keeps helm upgrade and helm uninstall commands consistent. Do not pass --create-namespace; the chart creates the namespace itself.

 

--set image.tag is not needed. The chart uses the version stamped by the Matrix42 release pipeline (Chart.AppVersion), which corresponds exactly to the --version you pass. Only specify image.tag if you are deploying a custom build.

What happens during installation

  1. Helm creates the tmc-system namespace and all RBAC (ClusterRoles, ServiceAccounts, ClusterRoleBindings).
  2. Helm creates the Harbor pull secret and copies your wildcard TLS certificate into tmc-system.
  3. Helm deploys the Argo Workflows controller and server into tmc-system.
  4. Helm deploys the application pod. Its init container waits up to 5 minutes for the Argo controller to become available before the main container starts.
  5. On startup, Matrix42 Tenant Management applies all pipeline WorkflowTemplates to the cluster, creates the bootstrap admin ServiceAccount (tmc-user-admin), and opens for HTTP traffic.

Verify everything is running:

kubectl get all -n tmc-system

All pods should reach Running / 1/1 within a few minutes (the init container wait accounts for most of the delay).

First Login

Generate the bootstrap admin token (valid one year):

kubectl create token tmc-user-admin -n tmc-system --duration=8760h

Open https://tmc-ui.esmp.company.com (using the hostname and domain you configured), paste the token, and click Log in.

Post-Install Configuration

A few things must be configured in Tenant Management application Settings before you can install a tenant. The Dashboard readiness badges show the current state of each.

1. SQL Servers
Open Settings → SQL Servers. Add at least one SQL server — give it a short name (e.g. prod-sql) and paste the full connection string. The connection string is stored as a Kubernetes Secret in tmc-system and never leaves the cluster in plaintext. You can add as many named SQL servers as you have servers; the install drawer lets you pick which one to use per tenant.

The connection string format:

Server=sql.company.internal;User Id=tmcuser;Password=secret;TrustServerCertificate=True;

The Initial Catalog (database name) is computed automatically from the tenant name during installation — do not include it in the connection string here.

2. S3 Object Storage
Open Settings → S3. Fill in endpoint, bucket name, access key, and secret key. Use the eye icon to verify what is saved. Saving a blank secret key keeps the existing value (useful for updating only the endpoint or bucket without re-entering credentials).

3. Loki Log Shipping
Open Settings → Loki. Fill in the endpoint URL and optional credentials. Basic auth (username/password) is supported.

4. Prometheus 
TBD

Readiness gates

The Dashboard shows a Ready / Not ready badge for each of the three services. The Install New Tenant button is disabled until all three are green. This is enforced on the server side as well — submit attempts with missing configuration are rejected with a clear error.

Operations

Installing a Tenant

Open the Tenants page and click Install New Tenant.

Provide the following information:

  • Tenant name — Lowercase DNS label, letters, numbers, and hyphens only (e.g. acme). Becomes the Kubernetes namespace and the URL subdomain. Cannot be changed after creation.
  • ESM version — Select from versions available in Harbor.
  • SQL server — Select from the servers you configured in Settings → SQL Servers.
  • ESM certificates — The ZIP file provided by Matrix42 for this customer.
  • Resource limits — CPU and memory for the controller, web app, and dispatcher. Sensible defaults are pre-filled; change only if you have specific sizing requirements.

Click Install. The pipeline starts immediately and you are taken to the run detail page to watch progress in real time.

What the install pipeline does

The pipeline runs these steps in order (some groups run in parallel):

  1. Create namespace + RBAC — creates the tenant namespace, ServiceAccount, Role, pull secret, and RoleBinding.
  2. Copy TLS certificate — copies the wildcard cert from tmc-system into the tenant namespace.
  3. Copy ESM certificates — stages the certificates ZIP into a temporary Kubernetes Secret in tmc-system, then copies individual certificates into the tenant namespace. Filenames are sanitised to satisfy Kubernetes Secret key constraints.
  4. Generate security material — entirely inside the cluster: generates a cryptography key, installation ID, worker token, client/audience secrets, and hashes the generated tenant admin password. Nothing is transmitted outside the cluster.
  5. Create tenant secrets — assembles all generated values plus the SQL connection string into a tenant-secrets Kubernetes Secret in the tenant namespace.
  6. Create SQL databases and schema — runs the ESM setup scripts against your SQL Server to create the databases and install the initial schema. On retry, this step drops and recreates all databases — this is intentional and expected.
  7. Deploy controller proxy — deploys the Linux-based controller service and its network policy.
  8. Deploy web application — deploys the IIS-based Windows pod, service, and ingress rule.
  9. Wait for web healthy — polls the web application's health endpoint until IIS is fully initialised. This step can take 1-2 minutes on first startup.
  10. Deploy dispatcher — deploys the Linux-based background job worker.

After installation

When the installation workflow completes successfully, a Retrieve Credentials button appears on the workflow run detail page.

The application does not retain the credentials after the credentials dialog is closed. However, the credentials remain available in the corresponding Argo Workflow run record until the run is garbage-collected.

Successful workflow runs are retained for seven days before garbage collection.

Accessing the Tenant

The tenant URL follows this format:

https://<tenant-name>.<domain>/wm

For example:

https://acme.esmp.company.com/wm

Once the tenant is running, click Open in the tenant table on the Dashboard to navigate directly to the tenant.

If an Installation Step Fails

Each workflow step automatically retries up to three times after a transient error, such as:

  • A temporary network interruption.
  • A temporary SQL Server timeout.
  • A temporary unavailability of an external service.

If the step continues to fail after the configured retries, the workflow enters the Failed state.

To investigate the failure:

  1. Open the workflow run detail page.
  2. Select the failed step.
  3. Review the complete step log.
  4. Identify and resolve the underlying problem.

If the workflow remains in the Failed state after all retries have been exhausted, delete the tenant namespace and start a new installation.

kubectl delete namespace <tenant-name>

Then open Install New Tenant again from the Dashboard.

Updating a Tenant

Open the tenant's detail page and click Update. The version picker shows only ESM versions in Harbor that are strictly newer than the tenant's current version.

What the update pipeline does

The pipeline is split into two phases separated by a mandatory human approval gate:

Phase 1 — pre-update (tenant stays online):

  1. database-check — runs the ESM update pre-check scripts against the current version databases.
  2. database-pre — runs the pre-migration scripts with the new ESM version.
  3. database-checkmain — verifies the main migration is ready to proceed.
  4. await-approval — the pipeline pauses here and waits for an operator to approve before continuing. The approval window is 7 days; if no one approves within that time the pipeline times out and must be re-submitted.

    Phase 2 — downtime (after approval):
  5. scale-down-web + scale-down-dispatcher — the tenant goes offline here. These steps run in parallel.
  6. database-mainandpost — runs the main database migration. This step is not retryable — if it fails, manual database recovery is required before resubmitting. This is the point of no return.
  7. patch-tenant-secrets — updates the product version in tenant configuration files.
  8. update-controller — switches the controller proxy deployment to the new image.
  9. scale-up-web — restores the web application at the new image version.
  10. wait-web-healthy — waits for IIS to initialise (same 1-2 minute window as during install).
  11. scale-up-dispatcher — restores the dispatcher at the new image version.

Approving the update

After the pipeline reaches await-approval, navigate to the run detail page. The paused step shows an Approve button. Clicking it confirms that downtime may begin. Only users with the Operator or Admin role can approve.

Typical downtime (from scale-down to web-healthy) is a few minutes for the migration itself, plus the IIS restart window.

Helm Upgrade

To upgrade Matrix42 Tenant Management and its bundled Argo Workflows to latest version:

  helm upgrade tenant-management oci://registry.m42cloud.com/esmp/tenant_management_helm --reuse-values

Or to a specific version:

  helm upgrade tenant-management oci://registry.m42cloud.com/esmp/tenant_management_helm \
   --version 1.0.9 \
   --reuse-values

 --reuse-values carries over all settings from your original install (harbor.*, tmc.*, tls.*) — no need to repeat them.

Pass the same --set flags you used during install if you need to change any values; omit them if nothing has changed. Helm remembers values from the previous install in the release secret, so unchanged parameters carry over automatically.

During the upgrade:

  • In-flight pipeline runs are unaffected — Argo Workflows operates independently and keeps running while the application pod restarts.
  • The new application pod applies any updated pipeline WorkflowTemplates automatically on startup.
  • The argo-workflows-workflow-controller-configmap is protected by a Helm hook that skips the ConfigMap if it already exists, so any S3 artifact repository configuration you have saved in Settings is preserved across upgrades.

If the TLS copy was skipped at install time (because the source secret did not yet exist), run helm upgrade after creating the secret to trigger the copy.

Helm Uninstall

helm uninstall tenant-management -n tmc-system

Uninstalling the Helm chart removes Matrix42 Tenant Management and Argo from the cluster but does not delete tenant namespaces. Existing ESM tenants continue running without a management plane. If you want to fully decommission everything, delete the tenant namespaces first.

 

Full cleanup


# 1. List all managed tenant namespaces
kubectl get namespace -l tmc.matrix42/managed=true

# 2. Delete each tenant namespace (this removes all ESM workloads and secrets for that tenant)
kubectl delete namespace <tenant-name>

# 3. Uninstall the Helm chart
helm uninstall tenant-management -n tmc-system

# 4. Delete the tmc-system namespace (removes Matrix42 Tenant Management, Argo, all cluster config secrets)
kubectl delete namespace tmc-system

Argo Workflows CRDs remain in the cluster after uninstall (standard Helm behaviour — CRDs are never deleted automatically to protect data). To remove them:

kubectl get crd | grep argoproj.io | awk '{print $1}' | xargs kubectl delete crd

Only do this if you are sure no other tooling on the cluster relies on Argo CRDs.

Authentication and Users

Matrix42 Tenant Management authenticates users with Kubernetes ServiceAccount tokens. There are no passwords stored in the application. Each user corresponds to a ServiceAccount in the tmc-system namespace; the token is validated against the Kubernetes TokenReview API.

Tokens are time-bounded. Generate a token for a user:

kubectl create token <service-account-name> -n tmc-system --duration=8760h

Pass the token on the login page, or as a Bearer token in API requests.

Bootstrap admin
On first startup, Matrix42 Tenant Management creates tmc-user-admin in tmc-system automatically. This is the account you use for first login and to set up other users.

Roles

  • Admin — All operations: configure cluster settings, manage users, install/update/decommission tenants.
  • Operator — Install, update, and decommission tenants; approve update pipeline gates.
  • Viewer — Read-only access to tenant status and run history.

Manage users on the Users page (left sidebar). The token for each user can be copied directly from that page.

Troubleshooting

Application pod not starting

kubectl describe pod -n tmc-system -l app.kubernetes.io/name=tenant-management-helm
kubectl logs -n tmc-system -l app.kubernetes.io/name=tenant-management-helm --previous

The most common cause is the init container timing out waiting for Argo. Check the Argo controller:

kubectl get pod -n tmc-system -l app.kubernetes.io/name=argo-workflows-workflow-controller

If the Argo controller is stuck, check that the harbor-pull-secret was created correctly:

kubectl get secret harbor-pull-secret -n tmc-system

Ingress not routing traffic

Confirm the nginx ingress controller is installed and its pods are running:

kubectl get pod -n ingress-nginx

Confirm the ingress resource was created:

kubectl get ingress -n tmc-system

TLS warnings in browser

The TLS secret copy is skipped if the source secret did not exist when Helm ran. Create the secret in the source namespace, then:

helm upgrade tenant-management oci://registry.m42cloud.com/esmp/tenant_management_helm \
 --version <current-version> --namespace tmc-system

Pipeline step stuck or failing

  1. Open the run detail page and click the failing step to read the log.
  2. Check that the relevant external service is reachable from the tmc-system namespace (SQL Server, S3, Harbor registry).
  3. After fixing the root cause, delete the tenant namespace and submit a new install from the Dashboard.

For the wait-web-healthy step specifically: IIS startup inside the Windows container can take 15–20 minutes on first start or after a restart. This is normal. The step retries on error with exponential back-off. If it fails permanently, check that the Windows node has enough memory (minimum 16 GB RAM recommended).

Checking what WorkflowTemplates are installed


kubectl get workflowtemplate -n tmc-system

You should see install-tenant, update-tenant, and all their sub-templates. If any are missing, Matrix42 Tenant Management failed to apply them on startup — check the application pod logs for errors from PipelineDefinitionService.

ESMP Containers Architecture Reference

Overview

Platform runs in immutable Docker containers managed by Kubernetes.

It is decomposed into a set of containerized services and workloads. Each component is packaged as one or more Docker images and distributed through the public container registry hosted in Harbor.

Container Types

The following sections describe the main container types used in the containerised architecture.

ESM Services Container

The ESM Services container hosts the M42Services web application. It provides:

  • The REST API endpoints for the ESM Service Layer.
  • The frontend assets required to render the HTML-based Single-Page Application (SPA).
  • Temporary Windows Functional Worker

The ESM Services container currently runs on a Windows Kubernetes node and uses Microsoft IIS as its web server. As part of the ongoing modernization roadmap, it will be migrated to Linux nodes.

Currently, each tenant requires a single instance of the ESM Services container. As part of the future High Availability initiative, this limitation will be removed, allowing multiple ESM Services container instances to run for the same tenant.

Functional Worker Container

The Functional Worker container hosts the Functional Worker engine responsible for executing background tasks and other ESM workloads. Functional Workers support horizontal scaling. Multiple Functional Worker containers can be deployed to process workloads in parallel.

The following worker configurations are supported:

  • Linux Functional Worker — the standard worker configuration used for the majority of background tasks.
  • Windows Functional Worker — used for tasks that require Windows-specific or “Legacy” capabilities.

Warning: Until the ESM Services container is migrated from Windows to Linux (Milestone 2), the Windows Functional Worker shares the same container with the ESM Services application.

 

As an alternative to deploying Windows nodes in the Kubernetes cluster, Windows-specific workloads can be executed by a Remote Worker running on a dedicated Windows Server virtual machine. The Remote Worker can be configured with the required Server and Workflow capabilities. This approach allows the Kubernetes cluster to operate exclusively with Linux nodes while still supporting workloads that require Windows-specific functionality.

 

Setup Container (esm_setup)

The Setup Container is a runtime container that is started on demand to execute maintenance and administrative operations. The container is not continuously running. It is created when a supported maintenance operation needs to be performed and is terminated after the operation has completed.

Controller Container

The Controller is a lightweight workload container implemented in Python. It performs infrastructure-level operations required to manage the ESMP deployment and its associated Kubernetes resources.