Private SDK: Temporary File Storage

Discover effective strategies for implementing temporary file storage in your private SDK to optimize data handling and enhance performance.

+ More

Table of Contents

Advanced Temporary Storage Component Methods Lightweight Temporary Storage Component for Powershell Methods

In distributed ESM deployments, an application cluster may consist of multiple servers, virtual machines, or containers. In such environments, temporary files stored on the local file system cannot be relied upon, as subsequent processing may be executed on a different compute node where those files are unavailable.

Typical use cases include:

  • Staging files for the Import Engine (GDI)
  • Exchanging large volumes of data between background processing tasks running on different compute nodes
  • Sharing temporary artifacts between application components in a clustered environment

The Temporary File Storage service provides a consistent abstraction for:

  • Creating and storing temporary files
  • Accessing temporary files from any application node
  • Automatically cleaning up expired temporary files

The service automatically selects the most appropriate storage backend and communication mechanism based on the deployment model. Depending on the environment, temporary files may be stored in 

  • Local file system
  • Azure Storage Account
  • S3-Compatible storage

The underlying access method (for example, direct file access or REST API) is selected automatically and is transparent to consumers of the service.

Advanced Temporary Storage Component

Supported starting from version: 26.2
Interface: Matrix42.StorageService.Contracts.ITempStorageService
PowerShell global variable: -

Methods

Method Description
UploadAsync

Uploads a file and returns its newly generated URI. Uses the Rest API to upload the file content in streaming mode.

Parameters:

 

  • fileName(string): The original file name (metadata only).
  • lifetimeMinutes(int): How long the file should be retained, in minutes.
  • fileStream(Stream): The file content stream.


Returns: Task<string>: The URI containing the unique identifier of the uploaded file.

Task<string> UploadAsync(string fileName, int lifetimeMinutes, Stream fileStream);
Download

Downloads the content of a temporary file by its identifier.

Parameters:

  • fileId(Guid): The file identifier.

Returns: Stream: The file content stream, or null if the file does not exist.

Stream Download(Guid fileId);
 Download (by URL)

   Downloads the content of a temporary file by its URL.

Parameters:

  • fileUrl(string): The file URL.

Returns: Stream: The file content stream, or null if the file does not exist.

Stream Download(string fileUr);
GetMetaInfo

Gets the metadata for a temporary file.

Parameters:

  • fileId(Guid): The file identifier.

Returns: TempFileMetaInfo: The file metadata, or null if not found.

TempFileMetaInfo GetMetaInfo(Guid fileId);
Delete

Gets the metadata for a temporary file.

Parameters:

  • fileId(Guid): The file identifier.

Returns: TempFileMetaInfo: The file metadata, or null if not found.

TempFileMetaInfo GetMetaInfo(Guid fileId);

Examples

Use Temporary Storage in C# code

var storageService = ServiceLocator.Current.GetInstance<ITempStorageService>();
var downloadedFile = storageService.Download(sourceFile.LocationUrl) ?? storageService.DownloadFromWeb(sourceFile.LocationUrl);

Lightweight Temporary Storage Component for Powershell

Supported starting from version: 26.2
Interface: Matrix42.StorageService.Contracts.ITempFileUploadService
Powershell global variable: $M42_TempFileService

Methods

Method Description
Upload

Uploads a file to the Temporary File Storage and returns a unique file URI. The file is transferred using the REST API in streaming mode.

Parameters:

 

  • fileName(string): Name of the file stored in the file metadata..
  • fileContent(string): Content of the file to upload..
  • lifeTime(int): File lifetime in minutes. Optional. Default is 1 day (1440 minutes). Maximum allowed value is 1 week (10080 minutes)..


Returns: string: Returns the URI of the uploaded temporary file.

string Upload(string fileName, string fileContent, int lifeTime);
Upload From File

Uploads an existing file from the local file system to the Temporary File Storage and returns a unique file URI.

Parameters:

 

  • fileName(string): Name of the file stored in the file metadata..
  • filePath(string): Full path of the file to upload.
  • lifeTime(int): File lifetime in minutes. Optional. Default is 1 day (1440 minutes). Maximum allowed value is 1 week (10080 minutes)..

Returns: Stream: The file content stream, or null if the file does not exist.

string UploadFromFile(string fileName, string filePath, int lifeTime);
 Download by File Identifier

  Downloads the content of a temporary file using its identifier.

Parameters:

  • fileId(Guid): The file identifier.

Returns: Stream: The file content stream, or null if the file does not exist.

Stream Download(string fileUr);
Download by File URI

Downloads the content of a temporary file using its URI..

Parameters:

  • fileUrl(string): The file identifier.URI of the temporary file

Returns: Returns a Stream containing the file content, or null if the file does not exist.

Stream Download(string fileUrl);

Examples

Upload file from the PowerShell script

# Upload local file to Temp Storage and clean up in 30 mins
$TempFile=$M42_TempFileService.UploadFromFile("user.csv", "D:\temp\gdi\users.csv", 30)
# Store variable content to Temp Storage
$TempFile=$M42_TempFileService.Upload("user.csv", "some content")

Related Articles